Skip to content

Commit

Permalink
Asymmetrical Sample Apartment (#25)
Browse files Browse the repository at this point in the history
* Fix Browser to send and recieve Mouse and Keyboard data

* Add Camera Controls for Better Asym Control

* Add Null Check

* Update Tracked Alias in Bow Sample

* Shrink Display Res and Make Input Non Static

* Shirk Default Stream Size

* Add Apartment Sample and Update Change and Project files

* Adding Unity Versions to README & Cloud Docs

* Add Apartment Sample to README
  • Loading branch information
vmohan7 authored Sep 21, 2021
1 parent 2b24c98 commit 2e332d1
Show file tree
Hide file tree
Showing 184 changed files with 16,553 additions and 23 deletions.
18 changes: 18 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,24 @@ All notable changes to com.unity.renderstreaming package will be documented in t
The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/)
and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.html).

## [0.5.1] - 2021-09-20

### Added

- Added a **NEW** Sample : Asymmetrical Apartment for testing asymmetrical multiplayer
- Add CameraControls.cs to allow for the Camera to be controlled in 2D experiences
- Added Supported Versions to README : https://github.com/FusedVR/VRStreaming/issues/24
- Added enableKeyboardTouchControls to VRInputManager to control whether to use Camera Controls

### Changed

- Fixed HDRPVR Sample to reference correct Render Streaming Prefab
- Fixed Mouse & Camera Input from the Web Client to be correctly sent to Unity SDK using RemoteInput class
- Removed WebXR controls from client and instead send input to be handled by server
- Fixed Overriding Prefab Positional & Rotational Data : https://github.com/FusedVR/VRStreaming/issues/19
- Rotation from headset is applied in local rotation instead of global rotation


## [0.5.0] - 2021-09-13

### Added
Expand Down
Binary file added Images~/apartment.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
24 changes: 20 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,12 @@ For an overview of this package, please refer to this video tutorial on the Fuse
5. On the Browser, click "**Connect to Cloud VR Streaming Server**"
6. Once you see the video feed, click "**Enter Virtual Reality**"

# Unity Versions

Tested & Verfied with Unity 2019.4 LTS. Limited testing has been done with Unity 2020.3 LTS and seems to be working as intended there as well.

Please also refer to [Unity Render Streaming](https://github.com/Unity-Technologies/UnityRenderStreaming) for their list of supported Unity versions.

# Configurations

On the **Render Streaming Service** Game Object, there are a few configuration options to make it easier to customize your Render Streaming Game.
Expand All @@ -40,7 +46,10 @@ At this time, the WebXR Client sends the resolution required to render per eye b

# Cloud Support

Limited testing and Proof of Concepts have been done with deploying a standalone version of the Unity Render Streaming app onto a Cloud or Edge Service provider via either a Windows or Ubuntu Virtual Machine. More testing is required to be done to enable this support.
This project has been tested to work on AWS EC2 leveraging Nvidia's Gaming AMIs. See the following videos for documentation on how to setup your Unity VR Streaming project and get it deployed on the Cloud.

- Windows Server with Tesla T4 GPU Accelerated Graphics : https://youtu.be/UFsbQ8YlboU
- Ubuntu 18.04 with Tesla T4 GPU Accelerated Graphics : https://youtu.be/3p0tzqD-s-c

# GPU Recommendations

Expand All @@ -50,14 +59,14 @@ It is strongly recommended to utlize a NVIDIA GPU as these GPUs support Hardware

Controller Input is captured via the [A-Frame Tracked-Controls component](https://aframe.io/docs/1.2.0/components/tracked-controls.html) and then sent over the data channel to the Unity SDK. This data protocol is adapted from the Unity Broadcast system, which was also capable of sending Keyboard, Mouse, Touch, and Gamepad data back to Unity via the RemoteInput.cs script in Unity. As such, VR data is sent from the client to Unity as Data Array buffers, defined in bytes.

The first byte of the data Array Byffer refers to the input mode to determine how to parse the data as an id. The following IDs were reservered for Web Input by the Unity Render Streaming system.
The first byte of the data Array Byffer refers to the input mode to determine how to parse the data as an id. The following IDs were reservered for Web Input by the Unity Render Streaming system and integrated with their Remote Input class.

- ID 0 = Keyboard
- ID 1 = Mouse
- ID 2 = MouseWheel
- ID 3 = Touch
- ID 4 = UI (legacy)
- ID 5 = Gamepad
- ID 5 = Gamepad (currently not implemented)

ID 6 is used for all WebXR Input specific to VR. Within VR Input, we specify 3 different modes for sending data, which are:
- ID 0 = Positional and Rotational Data of the Headset and Hands
Expand Down Expand Up @@ -99,10 +108,17 @@ The first one uses the Unity HDRP Built In Template to show case streaming a sce

![High Definition Render Pipeline](https://raw.githubusercontent.com/FusedVR/VRStreaming/master/Images~/hdrpvr.png)

The second provided sample shows integration with [VRTK](https://www.vrtk.io/) to showcase how to utilize the input that is streamed from the WebXR client into Unity. Using this input, we can build an Archery Sandbox. Simmply pick up the bow and then grab arrows from behind your back to begin shooting. Please note that to you use this sample you will need to manually also import the following [VRTK Tilia Packages](https://www.vrtk.io/tilia.html):
The second provided sample shows integration with [VRTK](https://www.vrtk.io/) to showcase how to utilize the input that is streamed from the WebXR client into Unity. Using this input, we can build an Archery Sandbox. Simply pick up the bow and then grab arrows from behind your back to begin shooting. Please note that to you use this sample you will need to manually also import the following [VRTK Tilia Packages](https://www.vrtk.io/tilia.html):

- **io.extendreality.tilia.camerarigs.trackedalias.unity**
- **io.extendreality.tilia.interactions.interactables.unity**
- **io.extendreality.tilia.interactions.snapzone.unity**

![VRTK Archery](https://raw.githubusercontent.com/FusedVR/VRStreaming/master/Images~/archery-sample.png)

The third provided sample shows an example of creating a asymmetrical multiplayer experience, where up to 4 users can enter the application either in VR or in 2D as a laptop or mobile user to view an apartment. VR users will be able to teleport around using the enabled [VRTK](https://www.vrtk.io/) integration. Please note that to you use this sample you will need to manually also import the following [VRTK Tilia Packages](https://www.vrtk.io/tilia.html):

- **io.extendreality.tilia.locomotors.teleporter.unity**
- **io.extendreality.tilia.indicators.objectpointers.unity**

![Apartment](https://raw.githubusercontent.com/FusedVR/VRStreaming/master/Images~/apartment.png)
3 changes: 2 additions & 1 deletion Runtime/FusedVR.Streaming.asmdef
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,8 @@
"name": "FusedVR.Streaming",
"references": [
"GUID:40a5acf76f04c4c8ebb69605e4b0d5c7",
"GUID:f12aafacab75a87499e7e45c873ffab8"
"GUID:f12aafacab75a87499e7e45c873ffab8",
"GUID:75469ad4d38634e559750d17036d5f7c"
],
"includePlatforms": [],
"excludePlatforms": [],
Expand Down
2 changes: 1 addition & 1 deletion Runtime/Prefabs/Render Streaming Service.prefab
Original file line number Diff line number Diff line change
Expand Up @@ -72,4 +72,4 @@ MonoBehaviour:
gameID:
maxConnections: 1
playerPrefabs:
- {fileID: 3432969078835647384, guid: 234ee74f5f05b9541850ec55278bb3f5, type: 3}
- {fileID: 1305806924054047947, guid: 234ee74f5f05b9541850ec55278bb3f5, type: 3}
11 changes: 6 additions & 5 deletions Runtime/Prefabs/WebXR Streamer.prefab
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,7 @@ MonoBehaviour:
m_Script: {fileID: 11500000, guid: 88dc1ef0a0e7d1b4a897f095b852ab38, type: 3}
m_Name:
m_EditorClassIdentifier:
streamingSize: {x: 1350, y: 1500}
streamingSize: {x: 972, y: 1080}
cameras:
- {fileID: 7977754495293142443}
depth: 0
Expand Down Expand Up @@ -154,7 +154,7 @@ GameObject:
serializedVersion: 6
m_Component:
- component: {fileID: 7977754496362167397}
- component: {fileID: 3432969078835647384}
- component: {fileID: 1305806924054047947}
- component: {fileID: 7977754496362167396}
- component: {fileID: 8526365356649225057}
m_Layer: 0
Expand All @@ -171,7 +171,7 @@ Transform:
m_PrefabInstance: {fileID: 0}
m_PrefabAsset: {fileID: 0}
m_GameObject: {fileID: 7977754496362167402}
m_LocalRotation: {x: -0, y: -0, z: -0, w: 1}
m_LocalRotation: {x: 0, y: 0, z: 0, w: 1}
m_LocalPosition: {x: 0, y: 0, z: 0}
m_LocalScale: {x: 1, y: 1, z: 1}
m_Children:
Expand All @@ -181,7 +181,7 @@ Transform:
m_Father: {fileID: 0}
m_RootOrder: 0
m_LocalEulerAnglesHint: {x: 0, y: 0, z: 0}
--- !u!114 &3432969078835647384
--- !u!114 &1305806924054047947
MonoBehaviour:
m_ObjectHideFlags: 0
m_CorrespondingSourceObject: {fileID: 0}
Expand Down Expand Up @@ -212,6 +212,7 @@ MonoBehaviour:
m_EditorClassIdentifier:
local: 0
label:
enableKeyboardTouchControls: 1
VRCameras: {fileID: 0}
VRPoseEvent:
m_PersistentCalls:
Expand Down Expand Up @@ -401,7 +402,7 @@ MonoBehaviour:
m_Script: {fileID: 11500000, guid: 88dc1ef0a0e7d1b4a897f095b852ab38, type: 3}
m_Name:
m_EditorClassIdentifier:
streamingSize: {x: 1350, y: 1500}
streamingSize: {x: 972, y: 1080}
cameras:
- {fileID: 7977754496754579055}
depth: 0
Expand Down
2 changes: 1 addition & 1 deletion Runtime/Scripts/ApplyVRData.cs
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ public void ApplyData(VRInputManager.Source id, Vector3 position, Quaternion rot
if (index == id) //check if the data is from the correct source
{
transform.localPosition = new Vector3(position.x, position.y, -position.z); //note that z data is reversed on WebXR
transform.rotation = rotation; //apply rotation - note coordinate system change was already applied
transform.localRotation = rotation; //apply rotation - note coordinate system change was already applied
}
}
}
Expand Down
149 changes: 149 additions & 0 deletions Runtime/Scripts/CameraControls.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,149 @@
using UnityEngine;
using UnityEngine.InputSystem;

namespace FusedVR.VRStreaming {

/// <summary>
/// Fly Camera Writen by Windexglow 11-13-10. And Modified for usage with VR Render Streaming
/// Original Source : https://gist.github.com/gunderson/d7f096bd07874f31671306318019d996
/// </summary>
public class CameraControls : MonoBehaviour {

#region Public Properties
public float mainSpeed = 10.0f; // Regular speed
public float shiftAdd = 25.0f; // Amount to accelerate when shift is pressed
public float maxShift = 100.0f; // Maximum speed when holding shift
public float camSens = 0.15f; // Mouse sensitivity
#endregion

#region Private Properties
private Mouse myMouse; // This Controllers Mouse Input
private Touchscreen myTouch; // This Controllers Touch Input
private Keyboard myKeyboard; // This Controllers Keyboard Input

private Pointer lastPointer; // For Mouse / Touch Input, what pointer was used last
private Vector2 lastPointerPosition = Vector2.zero; // The last position of the pointer
private bool isFirstPress = true; // whether the pointer has been previously pressed in the prior frame
private float totalRun = 1.0f; // speed control for the camera
#endregion

#region Public Methods
/// <summary>
/// Add a Unity Input Device (Mouse, Touch, Keyboard) to this controller to be used for Camera Controls
/// Used in the VR Input Manager to assign devices
/// </summary>
public void AddDevice(InputDevice device) {
switch (device) {
case Mouse mouse:
myMouse = mouse;
lastPointer = myMouse;
break;
case Keyboard keyboard:
myKeyboard = keyboard;
break;
case Touchscreen touch:
myTouch = touch;
lastPointer = myTouch;
break;
}
}

/// <summary>
/// Reset and Remove all Devices that have prior assigned
/// Used in the VR Input Manager to remove devices for the camera control on disconnect
/// </summary>
public void RemoveDevices() {
myMouse = null;
myKeyboard = null;
myTouch = null;
lastPointer = null;
}
#endregion

#region Camera Controls
void Update() {
// Mouse camera angle
if (myMouse != null && myMouse.leftButton.isPressed) {
if (isFirstPress) {
lastPointerPosition = myMouse.position.ReadValue();
isFirstPress = false;
}

UpdateMouseTouch(myMouse.position.ReadValue());
} else if (myTouch != null && myTouch.primaryTouch.press.isPressed ) {
if (isFirstPress) {
lastPointerPosition = myTouch.primaryTouch.position.ReadValue();
isFirstPress = false;
}

UpdateMouseTouch(myTouch.primaryTouch.position.ReadValue());
} else {
isFirstPress = true;
}

// Keyboard commands
Vector3 p = GetBaseInput();

if ( myKeyboard != null && myKeyboard.leftShiftKey.isPressed ) {
totalRun += Time.deltaTime;
p *= totalRun * shiftAdd;
p.x = Mathf.Clamp(p.x, -maxShift, maxShift);
p.y = Mathf.Clamp(p.y, -maxShift, maxShift);
p.z = Mathf.Clamp(p.z, -maxShift, maxShift);
} else {
totalRun = Mathf.Clamp(totalRun * 0.5f, 1f, 1000f);
p *= mainSpeed;
}

p *= Time.deltaTime;
transform.Translate(p);
}

// Updates the Rotation based on the Mouse 2D Position
void UpdateMouseTouch(Vector2 position) {
Vector3 delta = position - lastPointerPosition;
delta = new Vector3(-delta.y * camSens, delta.x * camSens, 0);
delta = new Vector3(transform.eulerAngles.x + delta.x, transform.eulerAngles.y + delta.y, 0);
transform.eulerAngles = delta;

lastPointerPosition = position;
}

// Returns the basic values, if it's 0 than it's not active.
private Vector3 GetBaseInput() {
if (myKeyboard == null) {
return Vector3.zero;
}

Vector3 p_Velocity = new Vector3();

// Forwards
if (myKeyboard.wKey.isPressed)
p_Velocity += new Vector3(0, 0, 1);

// Backwards
if (myKeyboard.sKey.isPressed)
p_Velocity += new Vector3(0, 0, -1);

// Left
if (myKeyboard.aKey.isPressed)
p_Velocity += new Vector3(-1, 0, 0);

// Right
if (myKeyboard.dKey.isPressed)
p_Velocity += new Vector3(1, 0, 0);

// Up
if (myKeyboard.spaceKey.isPressed)
p_Velocity += new Vector3(0, 1, 0);

// Down
if (myKeyboard.leftCtrlKey.isPressed)
p_Velocity += new Vector3(0, -1, 0);

return p_Velocity;
}
#endregion

}
}
11 changes: 11 additions & 0 deletions Runtime/Scripts/CameraControls.cs.meta

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

12 changes: 8 additions & 4 deletions Runtime/Scripts/ControllerInputManager.cs
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ namespace FusedVR.VRStreaming
/// The manager class for Controller Input Data i.e. Trigger, Grips Pressed, etc.
/// This class provides an abstraction over the raw data ControllerDataEvent from VRInputManager
/// </summary>
[RequireComponent(typeof(VRInputManager))]
public class ControllerInputManager : MonoBehaviour
{
#region Enums
Expand Down Expand Up @@ -71,18 +72,21 @@ public class VRAxisData : UnityEvent<Hand, Button, float, float> {
public VRAxisData VRAxisEvent;
#endregion

private VRInputManager input; // handles recieving the data

#region Methods
// Start is called before the first frame update
void Start()
{
VRInputManager.ButtonDataEvent += ButtonEvents; //start listening
VRInputManager.AxisDataEvent += AxisEvents; //start listening
input = GetComponent<VRInputManager>();
input.ButtonDataEvent += ButtonEvents; //start listening
input.AxisDataEvent += AxisEvents; //start listening
}

// OnDestroy is called when the game object is about to be destroyed in the scene
private void OnDestroy() {
VRInputManager.ButtonDataEvent -= ButtonEvents; //end listening
VRInputManager.AxisDataEvent -= AxisEvents; //end listening
input.ButtonDataEvent -= ButtonEvents; //end listening
input.AxisDataEvent -= AxisEvents; //end listening
}

/// <summary>
Expand Down
Loading

0 comments on commit 2e332d1

Please sign in to comment.