Open Source Platform
for interconnected virtual worlds

User Agent Model Working Group

From Rex community wiki

Note: Most things are written here from gameplay point of view, but many of these features should be interesting in their own right and not just as gameplay features.


Contents

Motivation

Current RealXtend architecture is based on the Second Life user agent model of one user agent, one avatar, one connection at all times. This poses some inflexibilities and requires non-ideal workarounds to implement certain features. Clearly a better architecture is needed to make development of new creative user agent-related features easier and more flexible.

Furthermore, tangetially related is a scene model of the world. OpenSim world is based on continous 2D-grid of regions which again poses some difficulties when trying to implement certain kind of worlds. A better scene model may be needed in addition to better user agent model to help create rich, immersive and distinctive virtual worlds with Opensim and the RealXtend viewer.

Below are some use cases which would be difficult to implement under the current architecture.

Use cases

Multiple controllable entities

Real time strategy (RTS) game where a single user can control multiple entities. The user can control only the entities that belong to her. The user can only initiate hostile actions to entities that are not controlled by her or her allies.

Multiple viewpoints into the world

Control center in-world, with a view to various other parts of the world (possibly even to completely different sims). The viewpoints are not under user control, but solely defined by the server.

Free camera

User switches to 'ghost camera' mode on the viewer. The camera is detached from the avatar and can be moved around freely. Any server-side view optimizations, such as object culling, work on the active camera.

Multiple cameras

The user opens multiple tabs on the viewer, with each tab showing a different part of the world via different in-world cameras. Any server-side view optimizations, such as object culling, work on all the cameras.

No avatar representation for the user

Game lobby, spectator mode. The user has no visual representation or (network replicated) position in the world. The user may still potentially move around in the world and region changes/camera optimizations should still work. For lightweight operation, collisions may not work.

Varied input methods for controllables

Pressing 'W' makes an avatar move forward, and 'A' and 'D' to strafe. For a vehicle, 'W' accelerates and 'A' and 'D' steers the vehicle to left or right.

Switching controllables

User is inside a dome as a human avatar. Outside the dome are fishes swimming in water, controlled by an AI. User switches from normal human avatar into one of the fishes, turning it into a fish avatar. The human avatar remains behind, possibly controlled by an (server/client) AI from that point on. When user switches back to her human avatar, the AI regains control of the fish avatar.

Tangential use cases

These use cases may not be strictly related to user agent model, but are interesting and worth considering none the less.

Disjoint and disparate regions

User enters a cave that is a separate instance from the 'world' instance. A game lobby region that has no real 3D representation, but may have 2D user interface representation in the server.

3D regions

A space flight simulator, where space and regions can span to any direction.

Implementation

To make development of new creative user agent -related features easier and more flexible, following features are needed: better control for gameplay features and remove dependency to a visible avatar. Allow for varied methods of controlling entities in the world without 'editing' the world. Controllable entities can be avatars, vehicles, primitives, cameras or any other kind of objects in the world. Support for various and different controlling methods (input) for the controllables. Improved control of special effects with multiple points of view into the world. Varied types of entities in the world, in addition to primitives. Support for entity-component model, serialization of entities, provided entity components are serializable, for network replication of entities.

Server

Varied input methods for controllables

Requires server support: Vehicle_EC may have different speed than avatar_EC. Otherwise unwieldy hacks are needed to control the avatar movement, such as setting the avatar speed from script when control changes to a vehicle mode and maintaining the state in the script. This may be possible at least to a point with a region module or with ModRex.

Each controllable contains a list of actions it can perform. These can be queried from the controllable. F.ex. MoveForward for avatar, Accelerate for a vehicle. The list may be implicit and common to both server and viewer, so protocol would not really need to know about the actions. However, a field in an update packet is probably needed to denote the type of action the controllable should perform.

Multiple controllable entities

Can't be done properly without server-side support: primitives can be moved by the client freely when editing but requires unwieldy avatar specific 'access rights' to restrict movement of some primitives, and to restrict how the primitives may be moved. It should be possible to turn a primitive into an 'agent', or controllable. Preferably with entity component support by having Controllable entity component in each controllable entity.

It may be possible to control multiple agents with ModRex by just using the 'AgentUpdate' packet, but there is still the problem that all controllables need to be agents. A way to 'upgrade' a primitive into an agent might be one option.

Multiple viewpoints/cameras

Currently support for one camera only and not used much yet server-side. Viewing a neighboring region through another camera not near the avatar yields artifacts. Needs support for other types of objects than primitives. May not be realistic if by design avatars are region specific, i.e. avatar can only be in one region at any one time. Currently multiple cameras in single region should work, as there are no camera-based optimizations in the server.

No avatar representation for the user

Avatar can be disabled client side depending on the scene. Camera updates are send in the same packet as avatar updates, they may need to be separated. Avatar, or avatar appearance should be separated from the user agent.

This may already be possible with ModRex.

Switching controllables

Avatar needs to be a more fluid concept. Any entity in the world may potentially be controlled by the user as if it was the user's avatar. User specific info, such as account / name, still need to be contained in one place but it should be separate from controllables. There still probably should be separate packets in the protocol for avatar vs. primitive (or controllable vs. editable) updates, since the controlling methods for the two may be too different.

Disjoint and disparate regions

There is at least two ways in which this is already possible. First is to have disjoint regions, where the user can be teleported between the regions, but where she may not directly wander between the regions. This is perhaps a bit unwieldy, especially if server sends all data and assets in the grid to the viewer. The two regions will have shared assets and scene presentation which may lead to poor performance in the viewer.

Second way is to have a separate grid and have the user teleport between the grids. This is the preferred way due to there being no performance penalty and each grid has their own assets and scene presentation. The problem with this approach is the lack of transparency of the teleport between grids as credentials may be required to be resend before the teleport. This may be smoother with the new viewer given certain restrictions.

Viewer

Multiple controllables

New EC_Controllable entity component. Possibly simply acts as a boolean for now, to determine if entity is controllable. May contain pointer to owner. A bit later may contain 'control actions' which can be performed to control it.

Separate user presence and avatar

Somewhat separate already, avatar is created when first AvatarUpdate-packet arrives to client. If no packet arrives, visual representation of avatar is not created. Create a 'Presence' class which supersedes / replaces current avatar class somewhat.

Multiple cameras / viewports

Somewhat supported in old viewer. Trivial to add multiple cameras in single region, as long as server uses no view based optimizations.

Server implementation plan

Note: this is currently just a proposal.

Varied input methods for controllables

When client logs in, server sends a message to client informin all controllable objects and their type (vechile, bot, avatar etc).

Each type is defined in their own module in Region server. These are custom RegionModules which implement the features of the controllables.

Implementing vechiles

Client sends "attach to vechile" message to server, server unhooks the regular agent update message handler and attaches it's own in vechile regionmodule. All inputs are now processed in that region module and converted to vechile controls. Avatar is also attached to vechile in pose of choise. For example to sitting pose in car, standing pose in Segway etc.

Multiple controllable entities

When controlling multiple entities like in RTS game, the objects wouldn't be selected like with vechiles. Instead the selection would be done in client and then new control message would be sent. Message format would be something like: List<UUID> controllables, string action, List<string> parameters.

Controllables would be the entities selected by the client (1-n items). Action would be some action defined in the controllables regionmodule, examples: move, attach, repair etc. Parameters is list of parameters that certain action has, it can for example contain coordinates in move, target in repair etc.

Viewer implementation plan

Generic

  • Rename EC_OpensimAvatar to EC_OpensimPresence to separate it more from avatar.
  • Add EC_OpensimControllable (or EC_Controllable). The component should be present on all controllables (agent). It should contain a reference to owner (or possibly a pointer to owner only if owned by that client, NULL otherwise)
  • Move appearance specific things from EC_OpensimAvatar to EC_OpensimControllable to separate it more from avatar.
  • Rename Avatar to Controllable. Possibly refactor / see which functions are relevant.
  • Move animation handling away from Avatar, to unify primitive and avatar animations more.
  • Change object update paths so that agent updates are send only when EC_OpensimControllable component is present and is owned by the client.
  • Allow all agents to be controlled by the user (in theory), but as above, only update entities with EC_OpensimControllable.
  • Refactor AvatarController and CameraController into EC_AvatarControllable and EC_CameraControllable.
  • In RexLogic, allow input to handle several controllables (but obviously only one controllable, the avatar, for now)
  • Replace the old way of handling input in controllers with the new improved way, with EC_AvatarControllable and EC_CameraControllable (and others) containing a list of actions the entity can perform, and then performing those actions based on input (previously actions were hardcoded and more or less common to all controllables).

Input methods for controllables

Proposal

Below is presented a method which allows for creation of controllables that can respond to various input independent of each other. For dynamic languages (Python, Javascript), only requirements are that the event system is supported and that component properties can be accessed and modified.

The basis for controllable is EC_Controllable component, which is present in all entities that can be controlled by the user. The component itself is a dummy container of data, it contains an ID for easier identification of different types of controllables, and a list of actions the controllable may perform.

Associated with the EC_Controllable component is ControllableHandler-class, which sends controllable action events based on the state of the component. This class is internal to the framework and need not be used directly.

With each type of controllable is a class (C++, Python, Javascript or any supported dynamic language) that accepts controllable action events and moves the controllable entity based on the actions. This class also accepts input events and changes the action state of the component based on the input.

In pseudo code, how to create a new controllable in C++, or in a dynamic language (Python, Javascript). 

class TankControllable:
    InitializeTank(Entity tank_entity):
        tank_component = tank_entity.GetComponent("EC_Controllable")
        tank_component.AddAction(move_forward)
        ...
        tank_component.AddAction(rotate_turret_clockwise)

    HandleEvent(Event event):
        if (event.category == Scene):
            if (event.type == EntityCreated and event.data.controllable_type == "Tank"):
                InitializeTank(event.data.entity)
        if (event.category == InputEvent):
            if (event.type == MoveForward):
                tank_component.SetAction(move_forward)
            ...
            if (event.type == RotateClockwise):
                tank_component.SetAction(rotate_turret_clockwise)
         
        if (event.category == ControllableAction and event.data == tank_entity):
            if (event.type == move_forward):
                tank_entity.MoveForward()
            ...
            if (event.type == rotate_turret_clockwise):
                tank_entity.GetComponent("Turret").Rotate()
Rationale

The above design has following strength:

  • No major viewer api requirements.
  • Relies on events and entity-component model, so can be easily used also from dynamic languages.
  • Due to using events, overriding and replacing controllable behaviour is possible. By catching 'ControllableAction' events one can implement their own controlling methods for controllables, or by catching 'Input' events, one can remap input controls for controllables.
  • 'ControllableAction' events can be sent from anywhere easily to control entities in predefined ways. Entities can easily be controlled by user input, AI, or any other method without having to do direct fine control for the entity.
Retrieved from "http://wiki.realxtend.org/index.php/User_Agent_Model_Working_Group"