CAcert System Admins discussion list

[Daniel Black <daniel AT cacert.org>]

well a few tips really.

Looking at the logs for crl.cacert.org I noticed there is a user fetching a 
copy of the crl every hour using wget. This user fetchs the same data at
 least twice and sometimes more because the CRL hasn't actually changed
 in that period.

TIP 1: timestamps

So tip one is to use the --timestamping (aka -N) option with wget. This checks
 the last modified time of the server and compares this to the existing files
 modified time before deciding to download it.

This would get a result like the following when use the second time:
wget  --timestamping -S http://crl.cacert.org/class3-revoke.crl
--2009-08-22 10:13:43--  http://crl.cacert.org/class3-revoke.crl           ;
Resolving crl.cacert.org... 213.154.225.236                                
Connecting to crl.cacert.org|213.154.225.236|:80... connected.             
HTTP request sent, awaiting response...                                    
  HTTP/1.1 200 OK                                                          
  Date: Sat, 22 Aug 2009 00:13:37 GMT                                      
  Server: TUNIX-httpscreen/4.0                                             
  Last-Modified: Fri, 21 Aug 2009 23:45:03 GMT                             
  ETag: "75001dc0"                                                         
  Accept-Ranges: bytes                                                     
  Content-Length: 245470                                                   
  Vary: Accept-Encoding                                                    
  Keep-Alive: timeout=15, max=100                                          
  Connection: Keep-Alive                                                   
  Content-Type: application/x-pkcs7-crl                                    
Length: 245470 (240K) [application/x-pkcs7-crl]                            
Server file no newer than local file `class3-revoke.crl' -- not retrieving.

Once this happens CAcert, you and every ISP in between has saved at least 50%
 bandwidth.

TIP 2: compression

Looking at options for wget you probably won't see an option to request 
compressed data. There is a way to manually put headers in and decompress 
using an external program however.

The following command line combines timestamping and compression. If for some
 reason the server doesn't deliver the crl in gziped format the gunzip will 
fail and you should have and uncompressed crl.

wget --timestamping  --header="Accept-Encoding: gzip" 
http://crl.cacert.org/class3-revoke.crl  && gunzip ;< class3-revoke.crl > 
class3-revoke.crl.new && mv class3-
revoke.crl.new  class3-revoke.crl

To check the crl:
openssl crl -in class3-revoke.crl -inform DER -noout -text | more

Once compression is enabled approximately 2/3 of the CRL size is saved in 
bandwidth.

TIP 3: download daily or weekly but not hourly

The benefit of grabbing a few extra revoked certificates by downloading hourly
probably isn't worth it. If revoked certificate checking is important to you 
use 
OCSP.

TIP 4: enterprise proxies

If a large network install a proxy server so you don't have too many clients 
trying to fetch the same large files and using up your bandwidth.

TIP 5: use OCSP in preference to CRLs where ever possible

OCSP is a mature technology that gives the revocation status of a certificate 
with a lot less bandwidth usage. It does add a little delay to the processing 
of the the certificate. The status here can be cached for a period of time 
though.

Have fun.

-- 
Daniel Black
Infrastructure Administrator
CAcert

Attachment: signature.asc
Description: This is a digitally signed message part.