Open Source Platform
for interconnected virtual worlds

User Interface Working Group

From Rex community wiki

Contents

Mandate

Provide the Viewer with a modern, customizable, visually attractive, 3D-engine integrable, user interface.

Problem

realXtend has an ambitious plan to provide a platform that will work on diverse set of hardware, from PC workstations, to consoles, to hand-held 3D-enabled devices. Moreover it strives to provide a "next generation" set of user interactions, that take full advantage on the novel input devices becoming available, that take full advantage of the 3D nature of the platform, and the rich context of a virtual world.

Examples of how interactions are changing away from a PC application centric universe:

  • applications: World of Warcraft, iPhone suite, Nokia maemo, XO laptop Sugar, Wii, etc.
  • input metrics: GPS, motion, multi-touch, haptic/force feedback, acceleration, orientation, pressure, heat, etc.
  • form factor: hand-helds, "Wii-mote"s, netbooks, etc.; with different screen sizes, resolutions, and view characteristics

Clearly we cannot program one UI that works well given all those criteria, so we must provide a means allowing the UI itself to change under new circumstances. We need something flexible, that assumes very little, and presents information in new and innovative metaphors, besides traditional windows, buttons, and text entry boxes.

Animated widgets in particular are visually attractive and increase user intuition and understanding.

Requirements

  • Cross platform: works equally well on windows, linux, mac.
  • Has a good documentation base with examples, reference and samples, a large user base, mature codebase that is open source.
  • Presents a high-level API that uses Scene Graph, Canvas, and Actors as abstractions.
  • Handle a wide range of input methods and languages, and has good support for i18n, l10n, a11y.
  • Allows much of the UI to be described in human-readable text form: XML/JSON layout description, scripting, or something similar.
  • The UI system integrates nicely with another 3D system, and renders to off-screen surfaces that are managed by our viewer application.
  • Render-on-demand, to manage the rendering when necessary.
  • The input system is ideally well decoupled from the UI library, and can inject events when necessary.
  • Has a multitude of UI events that can be hooked into, like "OnMouseDragged", "OnKeyPressed", "OnStyleChanged".
  • Has support common and advanced controls (Label, Button, Check box, Radio box, Text box, Drop-down list box, List menu, Scrollbar, Slider, Images, Datagrid, TreeView, Graph, Tabbed layouts, Calendar).
  • Support for CPU rendering for compatibility.


Current Situation

We have decided to make the viewer a Qt application, due to it's excellent ability to meet the above requirements. However, both Qt and the 3D Engine expect to be solely in charge of rendering, and with both API and hardware concerns, integrating the two is still an open question.

Plan 1

Instantiate but do not show() Qt widgets. Pump the event queue with transcribed events from OIS. Manually trigger expose events to draw off-screen. Copy drawn widget to Ogre3D texture.

Known Issues

  • Tooltips do no show or they are shown in wrong place.
  • TextEdit cursor does not work correctly in QLineEdit in Linux systems and stays in QTextEdit (after activating another component).
  • Mouse cursor does not change it state to correct one (if you example take it over QTextEdit).
  • Window frames are not seen. (Embedded dialogs -Qt demo shows the frames, so it must be possible to set by style)
  • Stacked widget-type does not work
  • Diffrent color on calender-widget (if widget is activated)
  • No main widget thats children everything that uses QtModule would be? If not so: this will disable the inheritance of Qt StyleSheets from single top widget is not possible
  • Not proper event-based redrawing?
  • Currently dragging does not work perfectly (we need those window frames).
  • Keyboard issues : Sometimes keys are stuck (they print same letter even that key release is done)
  • Keyboard issues: Backspace is too slow and currently if Python-module is used it will restart Python-module.
  • It is difficult to resize (grow) internal widget from corners
  • Cursor changes state changes generate blinking effect
  • Sometimes when clicking inside of render window and outside of render window mouse press down state is on in render window which results that widgets are moving when mouse is moved on render window.
  • Sometimes close() signal shutdowns whole application
  • QWebkit: Ajax text-input fields does not work correctly in internal mode.
  • QWebkit: Flash does not work, when QWebkit view is set into QGraphicsView (this is know bug in Qt).
Retrieved from "http://wiki.realxtend.org/index.php/User_Interface_Working_Group"