Server-Side Embedded Scripting

I'm starting to look very hard the corresponding server-side analog to the Emissary, the Coordinator, or Implementer or some other $10 word to be introduced in the future.

I'm pretty much committed to writing my own V8 wrapper on the browser --- I really don't think I have much choice there, but I'm not sure I need to go that way on the server side (though I did discover that the v8.net wrapper for .net seems to run fine in .net core).   Though I think I can use MicroSoft ClearScript on the browser side.  But on the server side I don't have to do this.  I've been looking at alternatives and node.js is pretty much out.  It's just kind of hot mess sandbox-wise.  It assumes it has full user-level system access, like any piece of code on the metal would.

However, I am looking very hard at deno.   It has the ability to be run as a secure sandbox (you must enable network and disk reading/writing explicitly), and I think I can extend it to do what I need to.   It was scale up and out as needed in the cloud.

Basically, it needs to be able to load and implement gRPC (and protocol buffer) bindings to the Emissary.  Also I think there needs to be able to load multiple "Coordinators" in each V8.  Maybe it's light enough where I can limit a v8 to only one manufacturer.  I know I can't get a way with this on the browser side just yet.  Anyway, it has to be able to call the API for that server level to implement the business logic for that type of server (kudo, world, entity, avatar--also world and avatar servers are also entity servers depending on their role in the scene).

Also, it does seem like it works just fine with wasm, which is where I ultimately want to go.   They're making lots of use of typescript, which is also where I want to go (if I must do JavaScript like coding), and with Rust, and so maybe I want to look harder at that.  I may have to write the bindings in Rust, but we'll see.

...

The more I look at this, it may be impractical to directly add what deno calls "bindings".  Perhaps this can be done via gRPC or AMQP with the network layer turned on.   In this case, each docker images would be pre-created and perhaps constrained with iptables, since we should know what code we need to handle -- we can roll this out and handle versioning by just shifting to a new docker and have the old one stand down after it goes idle.   At least we can start with this.   The API layer should be nicely abstracted so we can change how it works.

Looks like I seriously need to up my TypeScript game.

...

And after even more looking, deno isn't quite ready for prime time yet.   I think for initial implementations, I'm being a little paranoid.   Ultimately if it implements the API it can be run, and it can be sandboxed pretty well in docker.  Be it in node.js, or in python, or whatever.  True sandboxing perfection we can work on over time.

How the Servers Work

So, I've been thinking somewhat on how the servers actually are going to work.  I had though to them initializing and handling their security and bringing up the web services that happen at the top level (the fixed gRPC channels).  Though, except for the example content, I had always thought that implementors will simply add code to the system. 

I'm not thinking now that that's going to be best way to do this.   I've started thinking of the Emissary idea that I had for the client, and am thinking of possibly extending that to the server side.  I was never thinking that the server needed to be sandboxed like the client would, but maybe it would make sense to have a similar technology on the server (which as a starting name, I'm calling the Coordinator).   Basically the Coordinator would handle the server logic for one or more client side Emissary.   The servers would handle a lot of types of infrastructure natively that the Coordinator could take advantage of and make calls to though the APIs.

This all depends on the V8 engines and the Microsoft ClearScript engine working in .net Core (though this is where everything is going, it's likely to work).  The server would be a lot more capable of handling multiple servers and V8 threads to handle some scale-out on the server.

Even with an engine like this, other parts of the server should be written in a plugin-friendly way.  Where a set of C# assemblies that conform to the proper interfaces can be consumed by the server and loaded up at runtime.  The thought runs, that, for example, the world-persistence layer might be written to use Mongo, but a different persistence layer might be added which persists in, say, MSSQL.  So, have to think ahead about what things might need overloaded functionality which could be pluggable.

I've learned a little bit more about C# on webassembly.  What I understand Blazor does is that there's basically a webassembly version of an IL interpreter.   So C# assemblies are directly consumed by this interpreter.  C# is not compiled down into webassembly yet.  Still that might work.  It may not be the most natural language for this though.  Go and Rust are still in the running.  I'm obviously already ready for Non-Web Embedding, but I'm not sure what's been done for this so far.

Basic PKI (VRWorlds-CA) is feature-complete

Basic high level PKI is now functionally completed.

VRWorlds-CA basically seems to do what it needs to do.  It creates and manages root certificates and intermediates/signer certificates and exports encrypted files which can be used by the kudo servers (security servers) for VRWorlds and encrypted backup archives to keep offsite.

Lots of testing left.

The intention was that critical info should be encrypted, and that all communications should be using SSL and I didn't want to leave that to last, so i did it first.

Now, onward to building the server side components to assimilate the data produced by the CA.  Also need to work on rearchitecting the existing VRWorlds-3DOM library repository into basically a set of server common libraries, and a browser library and the beginnings of the basic scaffolds for the 4 backend server types (Kudo, World, Entity, Avatar).   I already have pieces of these scattered here and there and they all need structure. 

All of the server parts are .net core and the browser is whatever is needed to run in Unity.

Working on first hello-world functionality, which is to log into an Avatar and have it materialize onto a plane with various cubes and spheres in scene, whose movement are being manipulated by client-side emissary code.

Don't Use RSA

Interesting article on asymmetric cryptography...

Don't Use RSA

So, looks like I will be using Elliptic Curves with Curve 25519.   I'm not exactly sure how well TLS supports this.  I know that SSH does.  We micromanage the connections between the browser and t he server, and between servers, so likely they'll accommodate it.

That's why this whole subject is fraught.   I do hope I can get some real experts to look at all this while we're in the experimental stages.

CA Progress and Nuget

I've been making steady progress with the VRWorlds-CA project.  The idea is to build a CA/Cheap-HSM out of something like a Raspberry Pi, which would be Air-Gapped and kept in a safe.   The system would generate an encrypted archive which would contain all of the root certificate info (though not the root's private keys) and the intermediate signing certificates which would get imported by the kudo servers via a thumb drive, and encrypted offsite backups which would be kept in a secured location. 

I've got the whole flow down and determined what classes and methods are needed and what they have to do.  I just need to fill in the blanks now.

I'm looking at uploading the basic libraries onto nuget, so I can simply import them into the browser (in unity).   I've got the current VRWorlds-3DOM building in .net-core mode (which is how I want the servers to run so they can run in Linux docker).  But, Unity will need probably .net 4.6 -- I'll have to play with that.  I think it's easy enough to build multiple targets and nuget must have a way of dealing with this.

Data Plane

Seems like lots of big system are going with the new infrastructure components, the control plane and the data plane.   Haproxy now has a nice idea for a data plane API.

Anyway, I thought this was interesting.  We might look at this.   Heading out to spikecon this week, hoping to get some good work done on this project.   Working on the CA beginning infrastructure so I can actually get it so you can connect to the servers and log in your avatar.