Developing with the Mysterious ViroReact
Some Do’s and Don’ts of this Not-So-Documented Library
I recently used Viro for my bootcamp capstone project, and boy was I in for a rollercoaster of a time. I wanted to dive further into augmented reality as I thought it was showy and unique; I’ve also been highly interested in its capabilities. I stumbled upon the documents for ViroReact, thought they looked nice and were well written, did the quickstart tutorials, was impressed, and was persuaded by its simplicity to use this technology for my project.
After a day or two of trying to produce my own AR/VR content, I quickly realized that although the docs look neat and fresh the docs are not incredibly thorough nor up-to-date. Doing some google searches on Viro and issues I was having I found almost NO support, forums, articles, or outside help on Viro and felt left on my own to debug everything one step at a time.
One reason ViroReact is lacking in the help department is because in 2018 Viro went open source and stopped keeping their docs up to date. They have a GitHub where you can post known issues but it looks like developers from Viro stopped responding about a year or more ago and not many people are checking that account any longer.
Don’t get me wrong, there are some super cool functions you can do with Viro that are relatively simple. You just have to figure out the correct way to do them and what works. That’s why I’m here to help you figure this out ~
Iphone’s, VR, and Viro -
I tried doing the simple VR tutorial on the ViroReact docs; messed with it a bunch; did some reading, and the conclusion was the after iOS 13, for some reason, the gyroscope in the iPhones are not working. If you are trying to do VR, immediately or after restarting and a couple of seconds, the screen will spin around rapidly and will become unstable. For this reason, with no documented solution, I stuck with ViroReact’s AR functionality and ignored it’s VR capabilities.
Run and Build -
In ViroReact’s documents it states it is not compatible with Expo. This through me into a loop of trying to figure out how to run and test my app. When using the normal ‘npm start’ it launches a testbed app you must install on your phone from the app store called ViroMedia. Your app generates an ngrok code you must enter into the testbed. This was really tedious to use and would time out after a certain amount of time or requests to the server. To get away from this, ViroReact suggests using Xcode in its documents. Xcode is a large, 15gb apple program that will run and build your app. It requires a lot of research and understanding to use it to its full capabilities. Thankfully, not too much had to be done with Xcode to get it to run and build. You have to install ‘cocoapods’ into your app and select the Xcode file for iOS or android. The filename is <yourprojectname.xcworkspace> in the iOS or android folder. I used my iPhone and connected it via a USB cable to my Macbook but you can use a phone emulator to do this as well. To use your phone camera, though, you have to connect via USB.
To find more on integrating Xcode with your project check out the docs https://docs.viromedia.com/docs/integrating-with-react-native-projects
Integrating your own React Native app with ViroReact-
The simplest way, and the way I did this, was to start using React Native inside the app.js within your ViroReact app. Because of the React Native version that the docs say is required for ViroReact (0.59) you cannot use React Navigator. The app.js file is built out already, and upon launch, will display a screen to choose either the AR functionality or VR functionality. You can change this layout with your own React Native code. I made the following and used components within the app.js file
I also added several different ‘pages’ such as a login, signup, and build portfolio screen that are rendered using state and behavior.
What to know about ViroFlexView’s and ViroText-
ViroFlexView is ViroReact’s 2-Dimensional version of a javascript ‘div’. When using divs in javascript, I would commonly adjust the height and width, add a boxshadow or add a background color to make ‘boxes’ or ‘cards’ appear with information inside them. You can do somewhat of the same ViroFlexView. You can ONLY add 2D content such as a ViroImage(image) or ViroText(text) inside it. I added a background color with an opacity to give it a futuristic appearance. Something to note, you cannot adjust the border radius on the flexviews. It does not seem like you can use flexbox or display flex qualities, either, such as justify content inside the ViroFlexView. All the text I added inside the ViroFlexView had to be positioned so that all of the content was not on top of each other. This was particularly annoying when trying to add animations to the ViroFlexViews.
In regards to animating ViroFlexViews, this seemed to be impossible without wrapping the ViroFlexView inside of a ViroNode. Even though the documents say that ViroFlexViews were capable of animation, this was not true for me. A ViroNode is almost like a transparent, 3D ‘div’. You can place items inside of it. I was able to mess around with some animation and got the ViroFlexViews to do some cool things like change the opacity and adjust scale and position, but ultimately, when putting the ViroFlexView inside the ViroNode, all of the ViroText would lose its set position and be way outside the ViroFlexView.
ViroText has this odd appearance to it that seems very pixelated and grey. To fix this problem, I had to increase the fontSize to something very high. Then, I scaled the text down with the scale property. The result made the text crisp, clear, and its intended color. You cannot add text shadow to ViroText.
onDrag and onPress-
onDrag is a property that allows you to drag the object with your finger. There are two types of drags you can use: ‘FixedToWorld’ and ‘FixedDistance’. FixedToWorld will place the object as far away from the user as desired, finding the nearest plane by your finger to land on. With this, you can move an object really far away from the user without having to walk over to it, physically. FixedDistance will keep the object within a certain distance from the user. This was what I implemented because I did not want my ViroFlexViews, 3D objects, or 3DPortals to be very far away from me nor did I want them to be always located next to a wall or the floor.
onPress is a property used to delegate what happens when an element is ‘clicked’ or ‘pressed’ with a finger. I had some trouble getting it to fire every time I pressed my finger down. To ensure the function fired, when you press you must make more of a swiping left motion with you finger. I am not exactly sure why this behaves the way it does but it is a common issue amongst several threads.
Viro3DObjects-
These are the 3D objects within your app’s camera screen. You can place them within your HelloWorldAR.js file along with everything else that happens in AR like your ViroFlexViews and ViroText. You can download/make your own 3D objects as a .obj or a .vrx file or you can download the 3D objects ViroReact provide for you for free. Vrx files tend to work better. I tried using several different 3D objects I had downloaded and even purchased a couple. They would render on my app but would be all white without any color. Each 3D object needs its associated textures or materials usually as .pngs or .jpgs in order to be colored. Even though I did attach and require my textures to the objects, only the objects that ViroReact provided to me seemed to render textures.
You can animate the 3D objects with the animate property. You use the name associated with the 3D object ( you can create your own custom animation by registering an animation, or you can view the attached animations such as those from Viro) and use properties such as the name, run (boolean), loop (boolean), delay (boolean), amongst several others. FYI the names of most of all the animations associated with Viro’s 3D objects is ‘02’. You can find more on animation here ~
It is not required to put 3D objects or images inside of ViroNodes for animation.
Viro3D objects must also use a light source; the ViroAmbientLight or ViroSpotlight to render. It is kind of likek being able to see items in your room in the dark. You need light! You can put the light source on top of the objects or anywhere within your HelloWorldSceneAR.js file, really. You can customize the angle, intensity, color, direction, and other properties of the light source.
To make your 3D Objects cast a shadow, you have to set the property of the light source, ViroSpotlight castsShadow to ‘true’. Then, you have to use a ViroQuad, which is an invisible plane you can use to cast shadows on, such as in the image above. Make sure its property arShadowReceiver is set to ‘true’ as well. Position and rotate the ViroQuad so that it is beneath the objects that are casting shadows. For position and rotation (as well as scale) the first number in the array correlates to the x axis ( how far left or right is it ← →, or its rotation on the x axis; its width). The second number in the array the same but how it correlates with the Y axis. The third number correlates to its Z axis (its depth, or how far away from the user). Example, rotation = {[-90, 0, 0]}; the quad is tilted 90 degrees on its x axis so that it appears ‘flat’. According to the example, it is positioned directly at the originating source, which is my Image Tracker at position{[0, 0, 0]} (more on image tracking below). You should note, that the quad must be a MINIMUM distance below the object, image, flexview, etc. you want to cast the shadow. For me, the 3D objects are not casting a shadow unless I move the ViroQuad about an inch below the object which would cast the shadow UNDERNEATH my desk. This was not desirable. I instead used the ViroQuad to display the shadows from my ViroFlexViews since they were hovering above my desk already.
Image Tracking-
This is one of the coolest and easiest functionalities I found working with ViroReact. They made it super simple to take a photo of something that you want tracked; a logo, money, in my case a business card. Once you have the photo you crop the image to only be whatever you want tracked. You don’t want anything along the edges that could mess up your tracking. Once you do that, register the image marker as so -
ViroARTrackingTargets.createTargets({
"businessCard": {
source: require('./res/business-card.jpg'),
orientation: "Up",
physicalWidth: 0.09 / real world width in meters
}
})Here, I am calling the image being tracked, “businessCard”, requiring the source image, setting its orientation to “Up” so that it knows to track it from a face up position (my card is usually on my desk), and you have to set the physical width to how big it is is the real world. Once you do this, import ViroARImageMarker and use it in your render() — return() statement under ViroARScene
<ViroARImageMarker target={"businessCard"} onAnchorFound={() => this.setState({ runAnimation: true })}>This is setting the target to my business card and when it is found, it is running my 3D object animations. MAKE SURE in you app.js file, in your ViroARSceneNavigator that you set the number of images being tracked. Like so-
<ViroARSceneNavigator {...this.state.sharedProps}
initialScene={{ scene: InitialARScene }}
numberOfTrackedImages={1}
_exitViro={this._exitViro}
viroAppProps={this.state.viroAppProps}
/>What’s really cool about ViroReact is you can use a 3-Dimensional object such as a ball, a coffee cup, anything really, to be tracked. I did not get around to exploring this functionality but know it can be done. You needed to use a 3rd party ‘scanner’ app to scan your object.
Those were a few my many struggles with ViroReact along with some impressive functionality. Because there are hardly any forums or help out there I wanted to try and minimize a lot of debugging and trial and error for folks wanting to use this tech. Hope this blog post helps!!!
If you’re beginning to make a ViroReact app, I highly suggest doing the tutorials first on the documentation site.
https://docs.viromedia.com/docs/tutorial-ar
The GitHub to this project is https://github.com/kosmos02/portfolio-portal-frontend if you’d like to see more.
