There are many resources on Matomo’s main site about how to go cookieless, and the merits of local importing of logs with Matomo.
This post will help get you started with an aMiSTACX deployment that installs {WordPress or Magento}, and Matomo On-Premise Edition in order to help you leverage the server log analytics import feature.
Note: For this post we will use WordPress with Matomo for our examples; however, the same would apply for our deployment package of Magento Open Source with Matomo On-Premise.
Step 1. Select the WordPress w/ Matomo package and deploy on AWS
Step 2. During deployment in the CFT options, select deploy Matomo, and select Apache as the web server. [NGINX did not function correctly with Matomo’s python scripts, and is not approved or validated to work correctly with this particular aMiSTACX deployment; however, you can still select NGINX during CFT deployment, and attempt to get it working using your own resources. Let us know if you solve the issues.]
Step 3. Set up your domain DNS structure before installing both WordPress and Matomo. For example: https://example.com
Note: You can also make use of a hosts file.
Step 4. Install both WordPress and Matomo using their respective wizards. If you set up your DNS correctly, the URLs will be:
https://example.com #WordPress
https://example.com/matomo #Matomo
Note: Generally username and password will follow the same as the WordPress or Magento install. The database will take the application name.
Step 5. SSH to your instance, and should you want to automate Apache log uploads into Matomo, then go to the ubuntu system crontab, and remove the comment and restart the cron. /ect/crontab
59 * * * * ubuntu /home/ubuntu/log-import-full.sh
Note: The above is set for an import once per hour. For large sites, you may want to have a manual trigger.
Important! Every time you import, the data could be appended. Our import script is designed to avoid overlap. See more info below.
To manually trigger the upload from CLI:
sudo ./log-import-full.sh
Note: Apache log location:
/var/log/apache2/
Step 6. Verify output was successful.
Step 7. Log into Matomo and verify the import. Make sure you set the day of report correctly.
As you can see, we captured GoogleBots’ crawl in great detail. This level of detail is amazing to capture, and can help prevent bot attacks. We have observed about 75% of our traffic is malicious in nature, and when used in conjunction with A51 and Cloudflare’s WAF, you can create and automate an amazing bot shield.
Advanced Import Scripts
To automate log imports via cron, and still allow manual triggers, we have experimented with the following script:
#Cron for Apache2 Logs
sudo cp /var/log/apache2/access.log /var/log/apache2/access.log.bkup
sudo mv /var/log/apache2/access.log /var/log/apache2/access_log
cd /var/www/wordpress/matomo/
sudo python3 /var/www/wordpress/matomo/misc/log-analytics/import_logs.py --url=https://example.com/matomo/ --idsite=3 --recorders=4 --enable-http-errors --enable-http-redirects --enable-bots /var/log/apache2/access_log
sudo ./console core:archive --force-all-websites --url='https://example.com/matomo'
sudo rm /var/log/apache2/access_log
#Roll import
sudo cp /var/log/apache2/access.log.1 /var/log/apache2/access.log.bkup.2
sudo mv /var/log/apache2/access.log.1 /var/log/apache2/access_log
sudo python3 /var/www/wordpress/matomo/misc/log-analytics/import_logs.py --url=https://example.com/matomo/ --idsite=3 --recorders=4 --enable-http-errors --enable-http-redirects --enable-bots /var/log/apache2/access_log
sudo ./console core:archive --force-all-websites --url='https://example.com/matomo'
sudo rm /var/log/apache2/access_log
sudo service apache2 restart
echo SCRIPT IMPORT ENDThe script will import and make a backup, and also allow the daily roll-over of log imports. As indicated earlier, if you re-import data on top, it will not overwrite but append.
Note: You can use Cloudflare’s rules to restrict traffic to your /matomo installation path.
~Lead_Robot