CAcert System Admins discussion list

[Andre Klärner <kandre AT cacert.org>]

Hi Wytze,

On Mon 21.10.2013 17:23:23, Wytze van der Raay wrote:
> On 19.10.2013 20:31, Andre Klärner wrote:
> > What might be interesting in this case: are there any anonymized logs for
> > crl.cacert.org? I'd like to consolidate the logs to check for patterns in
> > the traffic, like reoccuring user agents, frequent requests from a limited
> > set of hosts etc. Maybe we could even smart up the delivery by clever
> > caching outside the firewall, or redirecting specificly annoying clients 
> > to
> > another host that can handle the load better or to a small CDN.
> 
> There are *very* anonymized logs, in the sense that the origin IP is always
> replaced by the internal address of our proxying firewall. But the user
> agent field and referer are still available in these logs. If you have
> some scripts available that would be useful to run on these logs, let me
> know, and I can run them for you and publish the results.
> The logfile format is:
> 
> LogFormat "%h %l %u %t \"%r\" %>s %b \
> \"%{Referer}i\" \"%{User-Agent}i\""                     combined

I would like to take a survey of the user agents that visit us regulary:

    cut -d\" -f6 $LOG |sort|uniq -c|sort -r

Maybe this should be repeated for the referrer-field (-f4) to see if there
any properties stored, that might help identify the frequency of some
clients (while I suspect that they are empty).

For determining the frequency from only the user agent I have vague idea
how to do this, but I think I have to test this first on one of my systems
before I can safely let someone else run this code ;)

Thanks and regards,
Andre

-- 
Andre Klärner
prospective CAcert Organisation Assurer

CAcert needs funding to keep its services running, so if you find our
project useful please consider donating. For more information see
http://www.cacert.org/index.php?id=13

CAcert braucht Geld um seine Dienste weiterhin betreiben zu können. Wenn
Sie also unser Projekt nützlich finden würden wir uns über eine Spende
freuen. Mehr Informationen finden Sie auf
http://www.cacert.org/index.php?id=13&lang=de_DE

Attachment: smime.p7s
Description: S/MIME cryptographic signature