collective.googleanalytics 1.0a1

Introduction

collective.googleanalytics is a Plone product used to pull statistics from Google Analytics and display them in a Plone site. It defines Analytics reports that are used to query Google and display the results using Google Visualizations. Reports are Zope objects that can be imported and exported using GenericSetup XML and modified on a site-by-site basis. The product currently provides a portlet that can display results of reports as well as a control panel for setting credentials and configuring settings.

Installation

To install collective.googleanalytics, add it to the eggs section of your buildout. If you are using Plone 3.2 or earlier, you also need to add a ZCML slug. Then re-run buildout and restart Zope.

Configuration

When you install the product from the Plone Add-ons control panel, a new control panel called Google Analytics will be added to your Plone site. In this control panel, you can set the e-mail address and password that Plone will use to access Google Analytics. You can also configure the amount of time, in minutes, that report results will be cached, reducing the need to query Google. Sixty minutes is the default caching interval.

Basic Use

After you have set your credentials in the control panel, you can begin using Analytics reports. First navigate to the page where you would like to display the report results. Where you place the portlet depends on your goals and the type of report you are displaying. For site-wide reports, your user dashboard may be the most appropriate place. For reports that monitor a specific page or section of the site, it may makes sense to place the portlet on the page that it references. Even if you place a portlet on a public page, the portlet will only be visible to users who have the “View Google Analytics Reports” permission, which, by default is assigned to Managers.

Next, place the portlet as you normally would, using the manage portlets page and selecting Google Analytics from the list of available portlets. In the portlet add form, you can set the title of the portlet, the profile the portlet will use and the reports it will display. Note that the title of the portlet is not the same as the title of the report, which is displayed inside the portlet. The profile is the Google property where the statistics that you want to access live. Which profiles you can select within the portlet depends on which profiles the Analytics account you provide to Plone can access. If you do not see any profiles in the dropdown menu, check to be sure that you have correctly set your Analytics e-mail address and password in the Google Analytics control panel.

You can select more than one report to display within a single portlet. Each report performs its own query to Google, however, so including many reports on a single page may cause a decrease in performance. To change the order of the reports within a portlet, see the section on Managing Reports below.

Once you have set the portlet title and profile and selected one or more reports, click save to add the portlet. When you navigate to the page where you assigned the portlet, you should see the results of your report.

Managing Reports

collective.googleanalytics ships with four default reports:

• Site Visits This Month: Line Chart

• Site Visits This Month: Sparkline

• Top 5 Pages This Month: Table

• Top 5 Sources This Month: Table

All of these reports are site-wide reports, meaning that they will produce the same results regardless of where they are placed on the site. It is possible, however, to create reports that return results based on the page where they are placed. For information about creating page-specific reports, see the section on Creating a New Report.

Reports live in a Plone tool called portal_analytics. To view and modify reports, navigate to the root of the site in the ZMI and click on the portal_analytics utility. Since they are standard Zope objects, reports can be copied, pasted, renamed, deleted, imported and exported using the Zope buttons beneath the list of reports. They can also be moved up and down in the list using the up, down, top and bottom buttons. The order of reports in portal_analytics controls the order that they will appear in portlets throughout the site.

Report Properties

Analytics reports are persistent Zope objects that store the arguments used to query Google and the options needed to display the query result as a Google Visualization. They store this information as properties on themselves. These properties can be set using GenericSetup XML or through the web in the ZMI.

These are the properties that the report object defines:

Title

The title of the report in the management interface. This is the title that that the user selects when assigning a portlet.

Description

A brief description of the report. The description is mainly for developer reference and never appears in the Plone user interface.

I18n Domain

The domain for translating the report.

Page Specific

A boolean value for whether the report results change based on the page where the report is displayed. This value is used to determine whether the report results should be cached for the entire site or on a page-by-page basis.

Query Metrics

A list of Google Analytics metrics to use in the query.

Query Dimensions

A list of Google Analytics dimensions to return in the query.

Query Filters

A list of filters to use in the query. Filters are defined as strings or TALES expressions that evaluate to strings in the format METRIC==VALUE, where METRIC is the name of a Google Analytics metric or dimension, VALUE is the desired value, and == is the appropriate logical operator.

Query Sort

A list of metrics or dimensions on which to sort the query results. Sort parameters are defined using strings or TALES expressions that evaluate to strings containing the name of a Google Analytics dimension or metric. In addition, the name of the dimension or metric can be preceded by a minus sign (-) to change the sort order from ascending to descending.

Query Start Date

The date to start querying for Analytics data. Both start and end date are defined using TALES expressions. See the section on Using TAL and TALES in Reports below for more information.

Query End Date

The end end date for the query to Analytics. It is defined in a TALES expressions.

Query Maximum Results

The maximum number of records that the query should return. It must be a positive integer, and 1000 is the default value.

Report Column Labels

The labels for the report columns. These are defined using strings or TALES expressions that evaluate to strings. Where they appear depends on type of visualization. For the Table visualization, for example, they appear as the table column headings.

Report Column Expressions

A list of TALES expressions used to calculate the values of the report based on the data returned by Google. After the query is performed, the report iterates over the rows of data returned and calculates the value of each column. Naturally the values available in these calculations depend on the dimensions and metrics found in the query. Each dimension and metric is converted to a variable name by replacing the colon with an underscore. For example, the value of ga:uniquePageviews is stored in the variable ga_uniquePageviews.

When writing column expressions, it is important to convert value of the variable to the appropriate type before attempting to perform calculations. Google returns all data, including numbers, as strings, so expressions must use the appropriate Python functions (int(), bool(), etc.) to convert to the correct type. Similarly, the result of the expression must be returned as the type that the selected visualization expects.

Report Introduction

A block of TAL code that precedes the visualization containing the report results. TALES expressions within this code have access to the normal objects described in the section on Using TAL and TALES in Reports. They also get two special Python lists called data_rows and data_columns. data_rows contains lists of the the evaluated column expression for each row of query data. data_columns is a convenience list that reformats the data in data_rows by column instead of row.

As an example, take a report that defines these columns:

python:str(ga_city)
python:int(ga_visits)

data_rows and data_columns might evaluate to the following python lists:

data_rows = [
        ['Seattle', 50],
        ['Portland', 25],
        ['San Francisco', 15]
]
data_columns = [
        ['Seattle', 'Portland', 'San Francisco'],
        [50, 25, 15]
]

If you wanted to display the total number of visits in the title, you could includes this TAL in the report introduction:

<h3>
        <span tal:replace="python:sum(data_columns[0])">
                [Total visits]
        </span>
        Visits
</h3>

Report Conclusion

A block of TAL code that follows the visualization containing the report results. See Report Introduction above for instructions on writing the TAL for this property.

Visualization Type

The type of Google Visualization to use to display the report results. This property can be set to the name of any of the default visualizations provided by Google.

Visualization Options

A list of options and values, in the format of TAL defines, that specify the options for the visualization. The available options depend on the type of visualization selected. As with the report column expressions, it is important that the option expressions evaluate to the data type that the visualization expects.

For example, the height of a visualization that accepts an integer height option could be set as follows:

height python:300

It may be helpful to think of Analytics reports as having four logical sections:

• Metadata about the report

• Query criteria

• Report definition

• Visulization settings

The first four properties–title, description, i18n domain and page specific–make up the metadata section of the report. These properties do not affect the results of the report or its presentation. Instead, they determine how it is listed and cached.

The query criteria section of the report is made up of all the properties that begin with the word query. These properties determine the query that is sent to Google to retrieve Analytics data. As a result, TALES expressions in this section of the report do not have access to the variables and objects that store the returned Analytics data.

The report definition is composed on the four properties that begin with the word report. This section of the report takes the data returned by Google and does the necessary processing and calculations. In other words, it takes the columns of data that Google provides, which correspond to dimensions and metrics, and maps them to the columns of data that the report defines. As a result, the report column expressions property has access to the metric and dimension variables, like ga_day or ga_visits, that represent the results of the query. The report introduction and conclusion have access to the data produces by evaluating the report column expressions in the form of data_rows and data_columns.

Finally the visualizaiton settings section of the report consists of the visualization type and visualization options properties. These properties are used to produce javascript that uses the Google Visualizations API to render the report data.

Using TAL and TALES in Reports

Many of the properties of the Analytics report object accept TALES expressions or TAL as their values. (For information about which properties accept TALES and TAL, see the section on Report Properties above.) All of the TAL code and TALES expressions have access to a standard set of Python objects and variables:

context

The object on which the current view is being called. In most cases, this is the content object next to which the report will be displayed.

request

The current request object.

date

An alias for the datetime.date function. It is used to create dates used in the query start date and query end date properties.

timedelta

An alias for the datetime.timedelta function. It is used to create time offsets for the query start date and query end date properties.

page_url

The relative URL of the current request. This is most commonly used in the query filters property for creating page-specific reports.

In addition to these objects, some properties have access to special objects and variables that represent the data returned by the query:

Report Column Expressions

These TAL expressions have access to variables that represent the values of each dimension and metric used in the query. The names of these variables are found by taking the name of the dimension or metric and replacing the colon with an underscore. For example, ga:exitPagePath becomes ga_exitPagePath. For more information about using these variables, see the section on Report Properties above.

Report Introduction and Report Conclusion

These TAL blocks have access to two special data structures–data_rows and data_columns–that contain the results of the evaluated report column expressions. For more information about using these data structures to perform calculations, see the section on Report Properties above.

Creating a New Report

Now that you are familiar with the properties that make up an Analytics report, it’s time to try creating a new report from scratch. In this example, we will first create a report that calculates and displays the site-wide bounce rate for the last seven days segmented by browser. After we get that working, we’ll create a second report that displays the same information for any given page on the site. Let’s get started!

1. Navigate to the root of the site in the ZMI and click on the portal_analytics tool.

2. Click the Add Google Analytics Report button.

3. We’ll give our new report the ID site-bounce-rate-browser-week-column, following the naming convention of the default reports. This naming convention is optional, but it helps to keep things organized. Then click the add button.

4. Click on the new report to edit it. Give it a title of Site Bounce Rate This Week By Browser: Column Chart and this description:

   This report displays the site-wide bounce rate for the last seven days segmented by the user’s browser. It is useful for gauging how effective our site’s new multimedia features are in each browser.

5. Leave the i18n domain as analytics, the default value. If we were going to translate this report, we might use the domain defined in our site’s theme product.

6. Leave the page specific box unchecked. This report is site-wide, so we don’t need to calculate the result for each individual page.

7. Now the difficult part: determining the arguments for our query. If we consult the common calculations page in the Google’s Dimensions and Metrics Reference (see the section on Where to Learn More for the link), we see that bounce rate is calculated as follows:

   ga:bounces/ga:entrances

   So, set the query metrics to ga:bounces and ga:entrances.

8. We also know that we want to segment our results by browser, so we’ll se our query dimension to ga:browser.

9. Leave query filters blank. We don’t need to filter the query results.

10. In the query sort box, type -ga:entrances. We want to sort by entrances so that we’ll be guaranteed to be shown the most popular browsers. The minus sign preceding the metric indicates that the sort should be in descending order.

11. In query start date, enter this TALES expression:

    python:today - timedelta(days=6)

    We could create absolute dates using the date() helper function, but it’s ususally more useful to create relative dates using the today variable and the timedelta() helper function. Common mistakes in this step include forgetting the python: prefix for the expression or trying to pass the number of days as a positional argument to timedelta() instead of passing it as a keyword argument.

12. In query end date, leave the default value, today. We could use the python: prefix, but it’s not required because a variable name by itself is a valid TALES expression.

13. In query maximum results, enter 5. Since we set ga:entrances as the sort value, this will show us results for the top five browsers based on entrances. We could, of course, increase this number if we wanted to see results for more browsers.

14. Now that our query arguments are complete, we can work on our report definition. We want our report to show us two things: the name of the browser and the bounce rate as a percentage. So, we’ll define two report columns. In the report column labels, enter these values on separate lines:

    string:Browser Name
    string:Bounce Rate (%)

15. In the report column expressions field, enter these TALES expressions on separate lines:

    python:str(ga_browser)
    python:int(100*float(ga_bounces)/float(ga_entrances))

    The first expression is fairly self-explanatory; it returns then name of the browser as a string. In the second expression, however, we have to do some calculations. Since Google returns all values as strings, our first task is converting the values to the appropriate format, in this case, floats. (Even though these values are always integers, we want to convert them to floating point numbers. Otherwise, Python will round down the result of the division.) Once we have our values in the correct format, we can divide them according to the formula provided by Google

    This division will yield a number between 0 and 1. To make the result easier to read, we’ll multiply the decimal by 100 to get a percentage and round it off to the nearest integer.

16. Leave the report introduction property blank. We’re going to display the title of the report as part of the visualization, so we don’t have to do it here.

17. In the report conclusion, we want to list the browsers with the lowest and highest bounce rates among the ones we are displaying. In the report conclusion field enter this TAL code:

    <div tal:define="browsers python:data_columns[0];
            bounces python:data_columns[1];">
        <p>
            Highest bounce rate:
            <strong tal:define="max_bounces python:max(bounces);
                max_bounces_index python:bounces.index(max_bounces);"
                tal:content="python:browsers[max_bounces_index]">
                Browser
            </strong>
        </p>
        <p>
            Lowest bounce rate:
            <strong tal:define="min_bounces python:min(bounces);
                    min_bounces_index python:bounces.index(min_bounces);"
                tal:content="python:browsers[min_bounces_index]">
                Browser
            </strong>
        </p>
    </div>

    In the first tal:define, we extract the browsers and bounces columns from the data_columns list. Then in the subsequent tal:defines, we determine the highest or lowest value in the bounces column and find the index of that value. Finally we set the value of the strong element to the value in browsers column that corresponds with the index we have determined.

18. Finally, we will set the visualization options for the report. In the visualization type dropdown, select ColumnChart.

19. In the visualization options field, enter these options in the format of TAL defines, one per line:

    height python:250
    is3D python:True
    legend string:top
    legendFontSize python:10
    title string:7-Day Site Bounce Rate

    These options are all aesthetic. Once you become familiar with Google visualizations, you can adjust them to fit your personal preferences. For a full list of the options available for each visualization, visit the Google Visualization Gallery referenced in the section on Where to Learn More.

20. You’re done! Click the save button in the ZMI. Then test out your new report on the site as described in the section about Basic Use.

Now that we’ve created a site-wide report for bounce rate, it’s easy to create a related report that displays the same statistics for a particular page.

1. To begin, navigate to the portal_analytics tool in the ZMI.

2. Check the box next to the report you just created. Then press the copy button at the bottom of the list.

3. Press the paste button. A new copy of the report is created with the ID copy_of_site-bounce-rate-browser-week-column.

4. Check next to the newly pasted report and click the rename button.

5. Change the ID of the new report to page-bounce-rate-browser-week-column and press Ok.

6. Click on the new report to edit it. In the title, replace the word Site with Page, and edit the description accordingly.

7. Check the box next to page specific. This will tell Plone to evaluate the the report for each page instead of caching it for the entire site.

8. In the query metrics list, deselect ga:entrances and select ga:uniquePageviews. Also, leave ga:bounces selected. If you’re not sure why we need to use ga:uniquePageviews instead of ga:entrances, consult the Google page about common calculations referenced above.

9. In the list of query dimensions, select ga:pagePath, and leave ga:browser selected. We’ll use ga:pagePath to filter the results of the query to just the current page.

10. In the query sort field, replace -ga:entrances with -ga:uniquePageviews to reflect the change in metrics.

11. Similarly, in the report column expressions, replace ga_entrances with ga_uniquePageviews.

12. In the query filters field, enter this TALES expression:

    string:ga:pagePath==${page_url}

    Recall that page_url is a convenience variable that is set to the relative URL of the current request.

13. In the visualization options property, edit the title of the visualization to read 7-Day Page Bounce Rate.

14. You’re done! Save your changes and try out your new report on your site. If you assign the report in a portlet at the root of the site and then navigate to interior pages, you should see the results change.

Defining Reports in a Filesystem Product

Any product that imports a GenericSetup profile can define Analytics reports. These reports should be defined in a file called analytics.xml in the product’s GenericSetup profile directory. The easiest way to generate the XML for a report is to create the report through the web and then export it.

For example, after following the instructions above for creating a new report, you could use the portal_setup tool in the ZMI to create a snapshot of the site. Then you could navigate to the analytics.xml file in the snapshot and copy and paste the appropriate XML into your product’s analytics.xml file.

If you find that you need to write the GenericSetup XML for a report by hand, consult the analytics.xml file in this product’s profiles/default directory for guidance. Keep in mind that any XML or XML reserved characters must be properly escaped.

Where to Learn More

Creating and managing Analytics reports requires knowledge of the Google Analytics API, the Google Visualizations API, and Zope and Plone technologies such as TAL and TALES. These are resources that you may find helpful in learning these technologies:

Credits

Changelog

1.0a1 - 2009-12-23

• Initial release