mbzi
Joined 9 years ago
Joined 9 years ago
I am curious on 1) the size of your teams, and 2) your opinion on how this would work with E2E product domain based teams (mixed skill sets) vs skills based teams e.g. User engagement team (front and backend devs covering sign on, emails, etc) vs Backend engineers (maintain all backend systems, etc).
2.5.6 Apps that browse the web must use the appropriate WebKit framework and WebKit Javascript.
This means they cannot add a new API standard on iOS. This is why WebRTC was hamstrung until November last year.
https://bugs.webkit.org/show_bug.cgi?id=208667 https://bugs.chromium.org/p/chromium/issues/detail?id=752458
If the browser uses WKWebView or UIWebView, you're right, you can "mock" an API by injecting JS, and through tunnelling you can recreate native functionality. Not if its SFSafariViewController. As someone who has done a lot of fiddling with JS to Obj-C/Java tunnelling for iOS and Android in production, it can be prone to errors and/or get your app rejected. Browser makers probably do not see the commercial value in going down this bespoke path.
"promising that we'll be able to build web apps in any language" is not how I see WASM, nor is it really used in this way outside of transpiling Unity3D/Unreal games (this may be the one area there is an exception). I use it to transpile C++ to WASM libraries used within React apps, Edge Lambdas, and Node.JS servers. Primarily down to 2 reasons: speed and efficiency. WASM unlocks excellence and resources in other disciplines/languages such as AI and AR tool chains, Engineers, OpenCV, etc. When used like this, it is outstanding.
What WASM will never be good at is being used for the whole experience. You lose the semantic web, and/or accessibility tooling. Web has some outstanding guidelines and frameworks to help the impaired, screen readers and the like. Using WASM to pump a native app into a HTMLCanvasElement will lose all of these advances, therefore, WASM shouldn't be used for this use case (outside of games). Like all tooling, there is a time and a place to use them.
Below are a few links which use WASM in production:
https://holition.com/play/holition-brings-home-twenty-awards...
https://winners.webbyawards.com/2020/apps-mobile-and-voice/a...
https://www.youniqueproducts.com/beautyguide#.YIS9hOhKguU
https://www.charlottetilbury.com/us/products/charlottes-virt...
If you were to implement a camera picker, take a gander at: https://webrtc.github.io/samples/src/content/devices/input-o...
iOS and Android don't have the same camera picker like Chrome on Windows 10, even then it is difficult to always override. I would recommend using enumerateDevices to allow you to pass the deviceId when firing getUserMedia. Not a lot of work. Implemented this a few times for cross-browser WebRTC experiences. A pro-tip is use getUserMedia with audio and video constraints set to true. This forces the permissions pop up to appear. You can kill the feed once the user has accepted the permissions, then ask them what device they want to use, and then fire getUserMedia with the correct deviceId. This is a [nice?] hack as some browsers don't allow you to use enumeratedDevices until after getUserMedia has asked you for the permissions. e.g. Safari...
https://developer.mozilla.org/en-US/docs/Web/API/MediaDevice...
WebRTC has been in iOS Safari for quite a while. However, all other apps and browsers were limited to what was in offer within WKWebView. From iOS 14.3 beta 1 onwards WKWebView gained getUserMedia/WebRTC features. It is now up to the browser and apps you use to make sure they have implemented their web views correctly and allow the correct permissions for WebRTC to work.
https://stackoverflow.com/questions/48775154/notreadableerro... https://bugs.webkit.org/show_bug.cgi?id=208667 https://bugs.chromium.org/p/chromium/issues/detail?id=752458
To allow surgeons to interact with my software within an operating room without the need for an assistant (to remain sterile).
Interactive retail displays outside the store. Users can interact with augmented reality displays and visualize themselves wearing the store products and/or to play a game to win prizes, etc.
Problems encountered:
Hardware adequate for long experiences e.g. Microsoft Life camera freezes after a few hours. Finding a device which can run 24/7 is a problem. Then once you found a good device you need to understand the risk of it being pulled from the market e.g. Primesense, Kinect, Intel RealSense (pulled and replaced by a new product and SDK, etc).
If a depth camera is used the type of bulbs to sunlight can interfere with tracking accuracy. If RGB is solely used then I am curious to see how well it works with various skin-tone in different lighting conditions and complicated backgrounds.
The "heavy arm/hand" problem. Try lifting your hand for 5 minutes and not putting it down. Users can be fatigued very quickly with a gesture based UX. Most products are not designed for this interaction.
In terms of Zesture:
The website is clean, to the point, great starting point. However I would like to:
- See an Enterprise license for long term support - Know how well it benchmarks against other SDKs/hardware solutions which achieve the same effect - Patents, does this infringe on other proprietary innovations? (do you have patent troll insurance?) - Guidelines for the best experience, e.g. distance from the camera if you were to use gestures to control a presentation - Roadmap, where are you going next?(FYI I am looking for a new way of hand based gestures which can be deployed via WebRTC and WebAssembly for interactive web based experiences :) )
Keep up the good work, looks promising!
This means products which help alleviate WebRTC infrastructure such as AWS Kinesis are not allowed (due to how they allocate turn servers with unknown IP addresses) and a company needs to manage their own infrastructure / TURN servers (which allows you to cherry pick where server locations are (HIPAA, country legal for what is streamed)) or accept Twillio's, or their competitors etc, large IP ranges (and don't have server location flexibility / increased commercial and market growth restrictions).
Whichever route you go down it is quite an undertaking!
P.s. Tsahi Levent-Levi is truly exceptional in this area. I highly recommend reading his blog and training courses: https://bloggeek.me/, https://webrtccourse.com/, AND he runs an amazing testing product https://www.testrtc.com. if you build your own infrastructure testRTC is a must.
Would I use this in production? No. But the author is not asking for that. It takes a lot of guts to create something and show it to this community. I really enjoyed looking at how the code progressed, how the author tackled a difficult subject, and their replies to constructive comments. Keep it up!
Without WebAssembly I wouldn't have been able to ship 2 products pro-bono within intensive care units and operating theatres directly helping with COVID.
I understand your dislike towards WebAssembly (albeit Web stack trends / flavour of the month esque development). I am not the largest fan of modern web development. Nevertheless love for WebAssembly is not due to developer productivity. After shipping 20+ WebAssembly products (alongside native counterparts) I am yet to meet an Engineer who enjoyed the WebAssembly/Emscripten/Blazor pipeline. However what WebAssembly has achieved for me is: Do people use your app? and within certain markets it allowed me to grow, do good, and say yes. This is the only real reason why someone should go down this route.
I think WebAssembly is more used than it appears, just difficult to see/tell.
A few years ago I actually tried integrating AR via WebAssembly with Amazon. We couldn't get the approval due to poor performance on Amazon fire devices (which have low end hardware). It is a shame but it is what it is.
What is disappointing/annoying is - as a CTO - it is near impossible to hire someone with WebAssembly skills. It requires an extra curious Engineer with a passion for native and web. Training is always important for a team but when going down the WebAssembly route you need to extra focused and invest more than what a typical Engineer would be allocated (E.g. Increase training from 1 day a week to 2-3). I suppose this may put people off?
Ten years ago I did the same but in Java and JOGL (before Apple banned OpenGL graphics within Java Applets embedded within a webpage). Was used for AR Watch try on within https://www.watchwarehouse.com and Ebay. The pain of Flash and Applets still wake me up at night.
I'm also building something very similar but with the ability for custom codecs (https://www.v-nova.com/ is very good). Probably the same issues too! Could I know more about your solution?