Cache/Config and Message Infrastructure

So, as I add on infrastructure I think I'm going with the following pieces, they should be deployed in a high-availability cluster configuration.

• Configuration information - ETCD
• Possible secrets storage - via Crypt, or possibly Vault
• Caching - Redis
• Messaging, Events and Subscriptions - AMQP via RabbitMQ

EtcD and Redis really do have different purposes and so it actually makes sense to have both.   I've used Redis, so I know it, as opposed to MemCacheD.  Etcd actually used gRPC internally so it kind of makes sense too.

I do need a place to store passphrases and keys.   This will likely go in EtcD, though encrypted themselves.  I'm not that fond of Vault, but I haven't used it in a long time -- I used an early version.  Once I get the PKI infrastructure working, I need to put it somewhere.

Kudo PKI infrastructure

So, now that I've gotten the basic infrastructure together for building our microservices with gRpc, I thought I'd sit down and start thinking about how the PKI and GUID identifiers work.   So, I cranked up the old Visio.

VRWorlds PKI and master Identifiers

Each manufacturer (entities doing VR work), has a single cluster of Kudo servers, any number of World clusters, and possibly one or more Entity and Avatar Forges with their attendant cluster of servers.   Worlds almost always have a dedicated Entity server to support it, but you can create Entities outside of Worlds too.   I envisioned that Avatar manufacturers would primarily be in the business to do that, but I can also see having Avatars, or at least Aspects (alternate bodies you can switch into), which were customized for a particular world.

I was thinking, at least initially, to build an inexpensive Raspberry Pi based HSM (which I'm sure makes any real security professional cringe and swear).  At the very first, this code will be run by hobbyists and developers.   Later this might become more complex and involve money and at that point, real security infrastructure and professional fiduciary care will need to be employed.  I want to try to build a good infrastructure behind the scenes to provide for future needs.   As I am just a humble developer working out of his metaphorical garage, and any code I write is provided "As-Is", with no expectation of real cryptological rigor. 

Anyway this PI would be kept air-gapped and be kept in a safe with a backup when not in use, and would generate and keep the root certificate and the signers, including future signers--likely using a thumb drive to transfer the data to the server (another cringe is heard).  I was thinking that keys would have a relatively short span -- a year or less, and perhaps keeping some power of 2 (4-16) actual intermediate signer certificates.  This reduces the metaphorical Blast Radius if a key needs to be repudiated.   Also they might be projected out a year or two into the future.  This way, a client will be able to prepare and handle the rollover transition from one certificate to the next as they expire.  That way they know the new public key is the same owner as the current one it trusts.

I do have some odd ideas when it comes to PKI and am adapting patterns from TLS and the web to other realms with some impedance mismatch between them.  But, I am not so stupid as to think I have the intelligence and skill with crypto to invent and build my own tools--that is the ultimate fool's road paved with good intentions.  I do want to use and understand what are supposed to be the best tools available for open source.  I also want to try to give a lot of attention to poking holes in my own infrastructure to make sure it's as strong as I can make it.

One of my conundrums is what happens when someone tries to poach/replay your GUIDs in their own fake server.   I think ultimately these will have to be combined with the public key and not assumed to be universally unique--though they're intended to be.   A kudo server can securely verify that it owns a public key.   Perhaps all 3rd parties must identify a server by a combination of the public key (or a uuid4 version of it) and the actual manufacturer/world/entity/avatar GUID--with the understanding that the public key rotates as it expires.   Initially I may even want the certs to expire every few days to force this rollover code to be well-tested.

The other big thing I want to start thinking about is how I will keep the secrets for this system.  At least things like keys should always be encrypted at rest and transit.  And of course, passphrases would be the same.   Kubernetes has a secrets infrastructure and docker sort of does.

Anyway, I'll start putting structure around this.  I'll likely prototype my hack-Pi-HSM (bash scripts and python are likely) and just run them with no expectation of actual security for now.  I am committed to all real communications with the servers forcing TLS on day one.

Next steps are boiling all this down to use-cases and working on what services need to be built were so I can start sketching out the servers.   All servers need to authenticate to their kudo server and come online.   Browsers need to log into the Avatar's kudo server.   Getting the avatar logged in would be a great foundation -- we can start dealing with meshes.

Milestone Accomplished - Ping Embryonic Kudo Server

Last checkin manages a ping.   I wrote a tiny client which calls the ping servers on the server via grpc with protocol buffers--proto3.   So, that's nice.  Hello Kudo Server!

I need to now build a real client and get several of the basic calls done as well as at least the Kudo, World, and Avatar.  Though might as well do the stubs for Entity too.   I also need to build myself a CA (certificate authority).   Initially I'll just build them via normal command line (or one of the simple command line CA packages).  I want the services to be SSL right off, and handle a proper authentication.   These can get sucked into a mongo database so the Kudo server can do the right thing.

Eventually I will build a Raspberry Pi to manage root certificates and generating the signing certificates.  Was thinking it would emit needed data onto a thumb drive.   This Pi should be kept in a safe.