CAcert System Admins discussion list

[Mario Lipinski <mario AT cacert.org>]

Am 07.08.10 12:35, schrieb Wytze van der Raay:
> Op 5-8-2010 5:22, Mario Lipinski schreef:
>> after the recent developments I do not expect CAcert to have additional
>> hardware available for running infrastructure during the next year. Since 
>> the
>> current situation does not give us any more resources for running 
>> additional
>> infrastructure (as I understood from dans mails), we have to see how we can
>> use our current hardware to be more flexible again.
> 
> Please note that the main reason for moving infrastructure services to
> hosting outside BIT/Ede was not to create flexibility, but rather to
> create a clear separation between critical services (to be audited) and
> infrastructure services (no auditing requirements).

Currently our only location where we have hardware available is BIT (we
currently have 5 good machines available). So we have to use it as good
as possible to provide all our services - critical and non-critical.
One goal of the proposed extension of infrastructure at BIT is to
restructure it so we just can take any service and move it to another
location. So it is preparing a distributed environment and also so
moving out of BIT in the longer term.

>> Taken from the documentation I have, currently sun1 and sun2 are running
>> infrastructure services. sun2 is somehow documented. What services are 
>> running
>> on sun1? I would assume sun1 to have resources left.
> 
> In fact sun1 is not functional anymore since some time, software-wise that 
> is
> (there's nothing wrong with the hardware). So all its resources are 
> available,
> in principle, but ...

So a new OS could be installed right now?

> Given the aging hardware of the current webdb server, and no resources in
> sight to acquire new hardware for it, we intend to move the webdb services
> to sun1 sometime in the coming year. While it wouldn't preclude alternative
> use of sun1's hardware in the short term, such alternative use should really
> be strictly temporary in order not to block this migration plan.

It would at least help us to rebuild infrastructure. E.g. it would be
good to run a 64 bit os which I guess the hardware is capable of.
Another alternative is to source another server. For infrastructure this
could very well be a used donated machine...
Guessing critical services cannot reduce their number of machines or to
be very offensive: The services on sun3 and sun4 cannot be put together?

>> So one possibility might be to move the running services off sun1, 
>> reinstall
>> it or (depending on the current setup) adjust the setup to run VMs in a to 
>> be
>> choosen way. For this it also should be considered for the future, to have 
>> a
>> distributed setup with VMs in different locations and to be able to move 
>> them
>> around. Backup should be an important thing either. If we are changing the 
>> way
>> we run VMs move them all to the new setup system, adjust setup of sun2 and
>> after that distribute them to spread the load.
> 
> It could be done I guess, but keep in mind that the use of sun1 for this
> migration should be temporary.

OK.

>> If this cannot work, we might need to consider to reduce the number of VMs 
>> by
>> installing nearly all services just on one host. This might reduce 
>> security,
>> but should give us some air to move again.
> 
> The use of separate VMs for each possible infrastructure service seems to
> have grown a bit out of hand I'd say. I guess you could start kicking out
> the ones that aren't really used.

So we really keep that in mind.

>> Another point we need to consider is, that the infrastructure at BIT 
>> currently
>> is tied to the critical services. When doing changes here, we should 
>> consider
>> to seperate these. This ideally would mean to have a seperate uplink for 
>> the
>> infrastructure subnet and attach it directly to the infrastructure boxes. 
>> If a
>> physical extra uplink is not possible we maybe could just simulate it. So 
>> no
>> firewalls which are also linked to the critical systems. Firewalling 
>> (packet
>> filtering) then has to be done on the infrastructure hosts - another factor
>> reducing security, but this should be acceptable.
> 
> A separate uplink is theoretically possible, but expensive, and won't bring
> any security benefits. The real issue in separation between critical and
> infrastructure systems is the currently shared use of firewalls, backup
> server and admin network. It's interesting suggestion to resolve that by
> simply moving the infrastructure systems away behind the firewalls and
> put them on the internet directly. That could be done indeed iff you
> accept the resulting reduced security for the infrastructure systems.

I would. I work with many hosted root servers which would fit for
infrastructure also. So it is quite the same situation. And we would
probably have it if we get a host donated somewhere. So the goal for
infrastructure at BIT should be for the short term to design it
independent from the critical stuff which is near there.
So an extra physical uplink is not economical, the firewall could simply
forward all traffic for the infrastructure networks to one interface.
Once there is an audit we can show that the systems are designed
independent and they can be moved out. And not having done so is just a
reason of the resources available by then.

>> After having restructured infrastructure at BIT, we can consider bringing
>> other single virtual hosts in. One topic here is backup: We need to be 
>> able to
>> bring a VM up at another location in a timely manner if something breaks.
>> Also after doing changes here, we should be able to move out of BIT (with
>> infrastructure - this is no plan for critical stuff) quickly, once there 
>> are
>> resources.
> 
> You will also need to plan some local backup -- the currently used backup
> server (part of critical systems) won't be accessible anymore in the "split"
> setup.

Yes. There is a netapp device in the equipment list which has never been
used for critical stuff. Could this host infrastructure backups?

>> So these are just some thoughts on infrastructure. Could this be realised? 
>> Any
>> other ideas?
> 
> See above ...
> I'm still amazed that CAcert Inc has found it prudent to turn down an
> impressive offer for external hosting of its infrastructure services
> sacrificing two valuable board members as part of the process :-(

Yes, that is hard. But we need infrastructure and we need more. So we
currently have to see how we can use all our resources available to make
the best from it. That is why I am trying to somehow shift available
resources around. I am aware that infrastructure needs attention from
many directions currently, so I'd like to start a process now, and not
like to wait until a machine turns up randomly.

-- 
Mit freundlichen Grüßen / Best regards

Mario Lipinski
Board member,                       E-Mail: 
mario AT cacert.org
Organisation Assurer (Germany),     Internet: http://www.cacert.org
Wiki/Issue admin
CAcert

Support CAcert: http://www.cacert.org/index.php?id=13
                http://wiki.cacert.org/wiki/HelpingCAcert

Attachment: smime.p7s
Description: S/MIME Cryptographic Signature