Parse logs for Matomo (Piwik) on Windows [using Powershell]

Automatic log parsing on Windows machines fot Matomo (Piwik).

Once again, thanks to: Adrian Ligiewicz (aka the Master of Powershell)

Remember to setup your start location in scheduler because script won’t find json file! Also change your paths according to your setup!


File: parse_logs.ps1
--------------------

#Based on https://matomo.org/docs/log-analytics-tool-how-to/

#------------------------------------------------------------------------
# Config

#:: Python URL (must be 2.x)
$PY_BIN="C:\Program Files\Python\Python27\python.exe"

#:: Matomo import_logs executable
$IMPORT_LOGS="[path_to_piwik]\piwik\misc\log-analytics\import_logs.py"


#:: IIS log dir
$LOGS_DIR="C:\inetpub\logs\LogFiles"

#:: Access token from Piwik config (check admin panel)
$TOKEN="*****************"

#:: URL to your Piwik installation
$URL="http://website.com/piwik/"

# CPU threads
$THREADS=2

#matomo.json extraction
$sites=(Get-Content -Raw -Path .\sites.json | ConvertFrom-Json)

#.json file example:
#[
#	{"DIR":"Directory1","ID":"1"},
#	{"DIR":"Directory2","ID":"2"},
#	{"DIR":"Directory3","ID":"3"}
#]



# Run command
foreach ($site in $sites)
{

# Automatically generated vars
$LOG_FILE="u_ex" + (Get-Date -UFormat "%y%m%d") + ".log"
$LOG_PATH="$LOGS_DIR\" + ($site).DIR + "\$LOG_FILE"


$PY_ARGs=" $IMPORT_LOGS --url=$URL --idsite=" + ($site).ID + " --token-auth=$TOKEN --recorders=$THREADS --enable-http-errors --enable-http-redirects --enable-static --enable-bots $LOG_PATH"
Start-Process $PY_BIN $PY_ARGs -wait
# Uncomment for debug:
# $PY_ARGs
Write-Host "Parsed log file: $LOG_PATH"
}

File: sites.json
----------------

[
	{"id":3, "dir":"W3SVC3", "name":"website.com"}
]

Same thing for dinosaurs (using bat)


File: matomo.bat
----------------

@echo off
:: Based on https://matomo.org/docs/log-analytics-tool-how-to/

:: Global config
SET PY_BIN=C:\Program Files\Python\Python27\python.exe
SET IMPORT_LOGS=[path to matomo]\piwik\misc\log-analytics\import_logs.py
SET LOGS_DIR=C:\inetpub\logs\LogFiles
SET TOKEN=****************************
SET URL=http://website.com/piwik/
SET THREADS=2



:: Change this according to your system locale. 
:: Current setting is based on dd.mm.yyyy format.
SET DAY=%date:~0,2%
SET MONTH=%date:~3,2%
SET YEAR=%date:~8,2%


:: Site 1
SET DIR=W3SVC3
SET ID=3

:: Zmienne generowane po dacie i ścieżkach
SET LOG_FILE=u_ex%YEAR%%MONTH%%DAY%.log
SET LOG_PATH=%LOGS_DIR%\%DIR%\%LOG_FILE%

"%PY_BIN%" "%IMPORT_LOGS%" --url=%URL% --idsite=%ID% --recorder-max-payload-size=200 --token-auth=%TOKEN% --recorders=%THREADS% --enable-http-errors --enable-http-redirects --enable-static --enable-bots "%LOG_PATH%"



@echo Parsed log file: %LOG_PATH%

Published by

Konrad Fedorczyk

Konrad Fedorczyk

I'm interested in programming and gamedev. I especially luv HTML5 and everything connected to web technologies.

Leave a Reply

Your email address will not be published. Required fields are marked *