On Meshes

Since most of the life of an emissary is to push reactive changes from the server into the manifestation of the Entity, which is mostly simply a group of meshes, I wanted to make sure there was a robust API for doing mesh manipulation -- somewhat beyond the functionality of Unity itself -- I'm looking hard at G3Sharp.  Which seems like an awesome library, and it's Unity compatible.

I need mesh alteration, composition and eventually mesh additive and subtraction operations.   So, some of this API will get wrapped and exported into the 3DOM APIs.   Also, you can have mesh files in your emissary payload (in several formats, including STL, OBJ), which can get loaded.

This is also part of the tooling projects I'm contemplating, which I'm calling the Wood-Shop tool.   I'm not much of a 3d artist, but I can craft pretty fancy 3d stuff in the real world, so I'm trying for a metaphor that's closer to working with wood and metal tools.

More Thoughts on VR Facing Applications

Riffing on that last post about Node/Chrome compatibility.   I would so love to be able to bring a fully functioning browser into VR, with an eye to being able to run chrome in Kiosk mode or even a version of Electron to be able to run Code or something like that inside.

However, unless it's amazingly easy to do, it's way down in the list of priorities.

However, I have been thinking of doing something like frame-buffer mapping as part of the emissary functionality.   I have my VT100 emulator, and there's a Unity VNC client somewhere (not sure how good it is), but ultimately it would be nice to have something like a X/RDP/Citrix functionality.  Also I envision lots of retro emulators, where something like this could expose the screen, possibly with point-audio.  Old school games have pretty low resolution, so not so hard.   RDP now can connect to one application on the remote side.

I don't know how this works into a virtualization/containerization world.   Certainly X works in a container.   Don't know enough about windows containers, though perhaps Wine might work too.

Thoughts on Emissaries and Chrome/Node Compatibility

So, I wasn't really thinking that the V8 environments in the browser would be truly compatible with node or chrome.  That leads to issues handling something like NPM compatibility.   Though I guess that is just what kinds of compatibility APIs I want to enable.  Some chrome compatibility shouldn't be too hard.

I'm wondering if it might be necessary for Emissaries to have dependency chains, and to have the notion of special Emissaries that simply implement libraries, like underscore.  Marshaling would then handle the import/export issues.    Of course, we don't have a DOM (though we might have a dummy DOM that we could do things with, perhaps even to the point of having a real DOM that could be displayed 2D in the world).   I've thought of doing odd things like porting QT (or even something as simple as TCL) to be able to run simple GUIs.  Really, supporting real web dialogs would probably be better.

I'm really just one person at this point and have to build in such a way that I can actually make progress, so obviously this is way out-of-scope for now, unless there are simple implementations that I can start with which can render into a texture in Unity.  I do need machine dialogs, and am thinking of the concept of an "App", which is a very special entity which is in your Active Inventory, which is something you can have as a HUD, or at least something you can bring up to interact with.

But, once I start writing Emissaries and their 3DOM interactions, I'll likely want a deeper environment than JavaScript, which I so hate -- I'm thinking maybe to do TypeScript instead, though I really would rather do webassembly, which is a bit of a mess right now as far as real Tool-Chains are concerned.  I can do C# (maybe), Go (maybe), and certainly Rust.

I have the hello-world, included in the box worlds and entity servers I wanted to build (A Virtual TARDIS), but still a bit up in-the-air as to what what platform I want to build them on.  They should be sufficiently standardized that I could build in C#, Go, Rust, Python, Java, or whatever.  I'm partial to Python and C# and want to do more with Go.   Rust and Java -- probably no.  APIs for them?  Maybe someday.

No Inherent Game or Movie Tropes

There are lots of tropes in games and in VR stories/movies that are designed to increase tension or add conflict to the story. 

• Your character can be killed.   Most games after you die you can respawn, with varying levels of penalty.  An extreme version of this is in 'Ready Player One', once you die, your character is dead forever.
• You lose your possessions when you die.  Sometimes they just get dumped there, or they might disappear.
• You have limits on what you can carry.
• You can only be in one game at a time.

It's possible to build games and have worlds where some of these things might happen within the confines of that world.  However, there are some basics which VRWorlds will enforce.

1.  There are two special types of certificates (special kudos), leases and deeds, which either give permanent or time-based access to Entities.   Avatars can only be deeded.   With deeds -- you own them.  Nothing that happens in any world will affect your ownership, or possession, of the object.   All this being said, your Avatar cannot die or be taken from you.   The only thing that can occur is that the original Entity or Avatar creator/provider can withdraw from the universe and take it's servers and your items with you.

2.  The rules about what you can lease or own are between you and the Entity server.  It may be that the Entity server allows you to own anything you pick up.  It may permit you to purchase/lease the item.   It also might be that the server won't let you take anything out of the world, but you will have it in your possession as long as you are in the world.  Items which you have deeds or leases for are yours and they cannot be taken from you, or even hidden from you.  They are in Active Inventory, even if you are not carrying them--you can call them back to you.

3.  There is no real limit on what you can carry of your own items.  A world might enforce carrying limits of world objects, but this cannot affect your objects.

4.  Worlds might not permit some of your objects to be used.  The Fluffy Bunny World might not permit you to use your 100 megaton nuclear hand-grenade, or they might.   Entities and Worlds have to cooperate on this.  There would have to be interlocked kudos to allow gameplay with interworld items.   I don't think third party worlds would want to trust self-declared attributes on a Entity from another service, but the world's physics engine would have to cooperate if an Entity wanted to explode, or shoot projectiles or otherwise be a weapon.  Your Avatar would have to agree to take damage, and there's no reason for it to even have a mechanism to do that, except for the purposes of gameplay.

5. There's no reason why your Avatar, at least notionally, can't be in multiple worlds at a time.  Now practically you can only function in one at time, but consider the analogy of having lots of tabs open in your web browser.   How you switch around, or get informed of interesting things happening in a given world are TBD, as is if your avatar will just stand there, or get ghosted out, or perform some activity via an AI.

6.  Kudo's are awards that are granted by worlds, or external groups (though I guess that can be a group of one Avatar).  They are a type of unreassignable crypto-coin.  Kudo's can be granted to avatars, worlds, entities (either the whole class of an entity, or individual instances), and so are owned by these objects.   Third parties cannot affect them, though the original grantor can revoke them.

Thoughts on Emissary Structure

Internal form of an Emissary Load File:

This is inspired by Android APK and Java Jar files.   But, I think instead of using a zip format, I will use a TAR format.   The reason for this is speed.   Also, after a possible compression layer is removed, the tar file can be internally seek'ed in order to emulate a high speed filesystem - while this is problematic for compressed files.   Since it is the intention of the Emissary file to become a read-only virtual filesystem for V8, this works rather well.   

Files at the top level (speculative):

• CERT.CA - Certificate of the code signer
• Manifest - A JSON file containing cryptographic checksums (sha256) of all the files in the module.   This should be signed by the above certificate.
• Manifest.sign - the digitally signed version of the sha256 of Manifest
• _code - directory containing the emissary code
• _code/main.js - the startup script
• _code/main.wasm (future) - a webassembly startup
• _payload - the root of the filesystem for the Emissary.  

Loading the Emissary is known as Marshaling it.

The system will open this file and inventory it (traverse the 512 byte TAR file headers and build an index) and store the index in a sqlite database.   If will check the certificate and load the manifest and check that it's properly signed.  Then it will checksum all the files and compare these with the manifest to validate the load.  Any errors will cause the Marshal phase to fail.   

If the original mime-type comes back as something like GZIP or a compatible compression type, it will be decompressed into a TAR file that is placed in a cache area (compressing these is recommended and may become mandatory, though we may ultimately support different algorithms).  That way, it will be available - already marshaled without needing to reload it.  However, new software revisions will have different codes and these will be automatically loaded.  Theoretically we can do hot code changes this way.   

Note on the read-only nature of the Emissary:  Besides the security aspect of allowing this to be modified, state is only allowed to be cached on the Emissary side -- we will likely give access to some small database or redis for this.  True persisted state exists on the server side, so we don't want to encourage the Emissary to think it owns state.   If we need to cache big Emissary-side objects (like big meshes) we'll make a way to do that inside 3DOM.

Initial Manifesto for VRWorlds

Welcome to my VR-Worlds Architecture Discussion

This is a rambling discussion of architecture decisions and other stream-of-consciousness maunderings as I pursue this project.   I started doing this as a series of markdown files, and discovered that it was rather too stream-of-consciousness to make a good file or FAQ (my brain bounces around too much).

VRWorlds (A poor placeholder name) is an architecture for a sandboxed browser for a federated virtual reality environment, which utilizes a similar paradigm as the modern web does for VR.  The idea is that worlds, avatars and the things which exist inside of worlds (entities) are provided by servers out in the web --and utilizes modern web service calls to implement them. 

It's almost a VR operating system, but it's federated like the current web.  It's the absolute opposite of a walled garden.   It's not designed for gaming (though there's no reason it can't be a fine gaming platform).  I want a place for my stuff, and I want to be able to work there (cover every surface with monitors and have working keyboards).  Anybody that wants to deploy servers can supply entities or avatars, or indeed make their own worlds.   And these can interoperate without whatever rules the creators want to implement.  There's no overarching policy to constrain it beyond the security considerations built into the browser.

The system is wrapped with a very modern security layer (all communications must be ssl/https, utilizing certificates on both the client and server side).

I do have some rather unique opinions about privacy and CA verification.   There is no external bureaucracy.   Each security server is the top of it's own tree--self-signed and free (of cost).  It can verify that it owns it's own public key (by signing something for you).  Bob's server will always be Bob's server, and thus has a reputational value.  It's not anonymous---it has a perpetual handle.  You don't have to have a certificate chain above it to tie Bob to the real world, though it's acceptable to do that if you like by doing your certificate through a more conventional provider.

There are 4 service layers to provide content, and these can be woven together in whatever complexity is required.

• World Server - provide location, layout (levels/rooms, etc) and some of the physics for a world.   Worlds can be inside other worlds (imagine a Tardis sitting on a starship--The starship is a world, and inside the Tardis is another, though the outside of the tardis is an Entity).  
• Avatar Server - The body and movement of the first person.   This also presents your third person to someone else in a scene in a world with you.  Avatars can contain other entities.  You might even have a tardis in your pocket.
• Entity Server - Objects inside a world.  Entities are made of a prototype, and an instance.  The prototype contains the generic functionality of the Entity, and an instance customizes it, at least to the extent that it has a real location in a world.  (A third person Avatar acts as an Entity, as would an AI or NPC).   Entities can contain other entities, and be moved from world to world (depending on the rules of the entity or world).   Entities can also be owned by Avatars.  As above, entities can be nested.  This technicality is important, because some entities sort of leave the world when this occurs.
• Kudo Server - The security layer that wraps all of the other services.  It's called this because my original idea was around reputational credits (a bit like a type of coin, though these were not tradable), a Kudo - an award granted by some authority, like Sarah's Virtual Flying School, might grant a Level 2 Flying Certificate.   In any case, this idea expanded to a full-on security layer, which includes things like a CA, and possibly LDAP and OAUTH2 authentication (with two-factor).   In this I am trying to utilize as many conventional open-source libraries as possible.  It's a fool's game to think you can invent your own security apparatus, and not end up with a giant bag of fail which can be exploited a million ways.  I'm not sure this isn't inevitable even using known libraries and tools--but as least there's action when the latest openssl exploit lands and I could update.  If it was in my code, I'd likely never know.

There's no real constraint on what platforms or languages the server layers may utilize.  I'm so far, thinking of doing c# core microservices.  

And then there is the browser itself:

The initial version will be written in C# in Unity.  It is intended that this be an actual VR application (I have a Vive), in SteamVR, but I want people to be able to run this in a regular display with a game controller in hand.   On the Vive I want fully articulated Avatars which can use traditional controllers or the new hand tracking technology (I have a Leap Motion).

So, the secret sauce to all this, is how I intend to make the browser work.  I can't afford to make it so that random foreign programs run inside the Unity APIs.   So, instead the browser will support a series of V8 environments running javascript and later (as soon as viable) webassembly code.  I call these codebehind/entitylets, Emissaries.

Emissaries are contained in something like a jar or apk file which contains a signed manifest.  They have a code and a payload component which allows the module to be started inside of V8 and the payload section attaches itself as a read-only filesystem to that code so it can have data payloads.   Emissaries communicate with a corresponding server component in one of the above servers (each entity/world/avatar type has a UUID, as does each instance of one).  The inspiration for this is the pub/sub mini-mongo environment inside of the meteor framework.  The idea is that the emissary basically is the client for a complex 3d object.  Instead of interacting with a DOM, it interacts with something I call a 3DOM, which is mostly an attempt as a secure wrapper for Unity APIs.

Lots of other parts of this, but I think this is a good initial manifesto.