It is difficult to see rules within the Observepoint interface. Users are required to scroll through pages to get past ten validation rules. It’s not easy to verify what tags are being validated in the UI.
We can solve this problem by generating the rule data associated with the Journeys from data in our PostgreSQL Database now that we have ETLs created to pull the data from the Observepoint API. We just need to query the data and assemble a report. Since our report will live in Atlassian Confluence, this Python script was created to generate wiki markup. We accomplish this task with a couple of functions. The first function showRules() displays the data in table format given a journey. It prints the journey info into a file.
cur = c.cursor()
# execute a statement
sql = (
"""
SELECT
distinct journeyid, jname, status, tags, lastcheck,
rulename, screenshot
FROM rules
JOIN actions
ON rules.ruleid = ANY (actions.rules)
JOIN journeys
USING (journeyid)
WHERE
journeyid = %s
"""
% journeyid)
#print(sql)
cur.execute(sql)
for rownum, row in enumerate(cur):
#header row
if rownum == 0:
print('----', file=f)
print('h2. ' + str(row[1]) + ' Journey ' + str(row[0]) + ' ' + row[2], file=f)
print('!' + row[6] + '!', file=f)
print('Last checked: ' + str(row[4]), file=f)
print('||Rule: ' + row[5] + '||', file=f)
print('||Variable||Operator||Value||Description||', file=f)
jl = json.loads(row[3])
acct = jl[0]['account']
variables = jl[0]['variables']
if variables is not None:
for v in variables:
rdesc = str(v['validationDescription'])
wikivalue = str(v['value'])
mapping = [ ('{', '{ '), ('|', '|') ]
for key, replacement in mapping:
wikivalue = wikivalue.replace( key, replacement )
if v['matchType'] == 'IsSet':
wikivalue = ''
if rdesc == 'None':
rdesc = ''
rule = '|' + v['variable'] + '|' + v['matchType'] + '| ' + wikivalue + '|' + rdesc + ' |'
print(rule, file=f)
#pprint.pprint(jl)
else:
print ('|No variables.| | |', file=f)
print(' \r', file=f)
The second function is used to query our database for the journeys we are interested in. They are organized in folders in Observepoint. It includes two table of contents wiki Macros – the second interesting because it calls out FAILURES. This function also opens our file and copies the contents to the clipboard for easy pasting into Atlassian Confluence.
fname = 'Production'
if folder == 'q':
fname = 'QA'
cur = c.cursor()
# execute a statement
sql = """
SELECT
distinct journeyid
FROM journeys
JOIN folders
USING (folderid)
WHERE
fname = '{0}'
""".format(fname)
#print(sql)
cur.execute(sql)
f = open('journeyrules.txt','a')
f.truncate(0)
f.write('{toc} \r FAILURES: {toc:style=list|include=Failure*|exclude=Ok*}')
f.write('report updated:' + str(datetime.datetime.now()) + '\r')
for rownum, row in enumerate(cur):
showRules(c,row[0],f)
f.close()
fo = open('journeyrules.txt', 'r').read()
pyperclip.copy(fo)
pyperclip.paste()
f.close()
print('******COPIED TO CLIPBOARD******')
Here’s the main function that asks the user which report they want to run. It also opens the report URLs based on the report the user asked for.
folder = input("Folder? p for production, q for QA:") # Python 3
if folder not in ['p','q']:
folder = 'p'
print ('running report for %s' % folder)
conn = None
conn = queryFunc.openConn()
getOP.loadJourneys(conn)
getOP.loadActions(conn)
getOP.loadRules(conn)
getOP.loadFolders(conn)
#printRules(conn,21745)
reportAllJourneyRules(conn,folder)
queryFunc.closeConn(conn)
if folder == 'p':
webbrowser.open('https://confluence.fds/display/MKTSYS/Production+Journey+Report')
else:
webbrowser.open('https://confluence.fds/display/MKTSYS/QA+Journey+Report')
Finally this is what our Atlassian Confluence Wiki Journey Report looks after a quick copy and paste:
