Glass for Captioning: First Field Tests

Glass for Captioning series:

Augmented Reality Captioning
Preliminary Impressions of Google Glass

First Field Tests

Last week I tried Google Glass out in the field for the first time. I've gotten a new pair now; Google was very helpful and accommodating when I told them about the optical flaw in my last pair and happily switched them out for a new one. There's still a little bit of glare underneath the screen, which they told me is pretty much inherent to the design, but it's much less than before, and the glare along the sides is gone, plus the smeary left edge has cleared up and the text overall seems crisper and less diffuse than before. I think my prior set just had a slightly misaligned projector or something. Absolute top scores to Google's customer service team, which was communicative, timely, and quick to move.

There are still some frustrations when it comes to actually using Glass for captioning. Initial tests seemed to offer Hangout Screen Sharing as a good solution; the resolution was clear enough that about 8 lines of captioning were visible, very readable, with all of my carefully tweaked Eclipse display and font options on view, plus it would allow me to use all the realtime editing tricks I rely on every day to clean up misstrokes and define new words from my steno machine. It sounded like a slam dunk. When I was just writing a few experimental words for myself, it seemed perfect. Unfortunately, as I feared, when I started actually transcribing the professor in action, the amount of lag involved in refreshing an entire computer screen several times a second quickly tanked the experiment. This was a particularly slow and steady lecturer, but the display was consistently 10 to 20 lines behind my laptop's display, and sometimes it would skip whole swaths of texts in order to catch up to the present, only to fall immediately behind again. So much for that idea. Incidentally, my laptop was on good quality institutional Wi-Fi, and Glass was tethered via Bluetooth to my phone, using the connection from that same institutional Wi-Fi. Glass can connect to most Wi-Fi networks directly, but this particular one required an authentication type that wasn't supported, so it had to piggyback from the connection on my phone. I've also had some trouble connecting it to my 4G hotspot, but I want to fiddle with it some more before deciding whether it's Glass's fault or user error.

So the next day I gave up the beautiful dream of lagless screen sharing and went to the fallback option: Hangout Chat. I set it up well before the class started, which was good, because currently Glass offers no way to turn off the little blips, bloops, swoops, and blats it makes while connecting to someone via Hangout. I really hope they offer a silent option soon; the bone conducting speaker makes the sound louder to the user than to anyone else in the room, but it's still clearly audible and potentially quite disruptive. By default, Glass displays video from whoever you're hanging out with, but I turned that off to save Glass's battery. So then it displayed my user icon, but that was a distracting background against which to view the text, so I set my user icon to a plain black rectangle. Then I muted Glass's own microphone and camera, also to save on battery life. This is what I wound up with:



The Hangout Chat interface without text.



The Hangout Chat Interface with text.

The two main distractors are the prefix of my username before every line of text that's sent (inescapable in this sort of text chat format) and the two prominent "camera muted" and "microphone muted" icons in the center of the lowest part of the screen. I think this is somewhat poor design, considering that new text starts at the bottom and is pushed upward, and that the very top of the screen isn't used by Hangout Chat at all. So rather than keeping the mute icons down at the bottom, interfering with the newest and presumably most important texts, why not put them at the top and out of the way?

The battery was also a little disappointing. On the previous day, when I was screen sharing, it died completely after less than an hour. That was too bad, but I had higher hopes for Hangout Chat, which presumably required less juice. And indeed, its total life was about an hour and 40 minutes, but the intensely irritating low battery alert came on just about exactly halfway through:




So not only does the alert pop up when the battery's presumably only at 50% of its capacity, but the alert is an entire line of full-sized text smack in the middle of the screen. How does that make sense? What's wrong with a discreet little battery icon tucked away in the corner of the screen? I can only hope that as the UI is updated (which happens on a pretty regular basis, I'm happy to say), this will be fixed to be less disruptive. I'll also probably post a comment to the Glass Explorers' Forum. This is, as I have to keep reminding myself, a prototype device, and a lot can change over the next few months.

But here's the most serious issue, which you can also see in the pictures above: Once an old line of text is pushed up to make room for a new one, it's suddenly severely truncated. So if you didn't manage to read the entirety of the text the first time around, it's going to be all but useless to you as soon as another line comes in. For captioning, where text can come in at a pretty solid clip, that is a big, big problem.




This is the main thing that's keeping me from enthusiastically offering Glass to my clients. If all they've got to work with is one line of text, it won't be good for much except slow-paced one-on-one conversations, and if the battery is really only good for less than two hours (I haven't yet tested it with the microphone active, which would presumably reduce the battery life even more, even though it would potentially allow them to interact and even move around without me and my machine having to sit there at their elbow), that severely restricts the circumstances in which Glass will actually be useful.

What about the future? Will Screen Sharing get less laggy? Will they remove the truncation from previous messages? Will they show the username the first time a message is sent and then allow it to be implied for subsequent messages? Will they condense alerts to icons and move them out of the way? Or will I have to commission special captioning Glassware to solve all these problems for me? I guess we'll have to wait and find out.

Oh, and one last thing. By request, a picture of me actually wearing Glass. Dork Factor: Significant.

Preliminary Impressions of Google Glass

For background, read my first post on augmented reality captioning.

I picked up my Google Glass last Thursday. It's certainly an impressive bit of hardware, and I'm very excited about the possibilities for captioning, but of course it's still a prototype device; the consumer models won't be released until after a year of additional quality testing and user feedback. The first pair they gave me had an unresponsive touchpad, and the second pair (the one I have now) seems to have some kind of optical defect that results in a lot of light scattering and glare, which I didn't notice with the first pair. I think I'm going to have to go back to Google to see if they can either repair the problem or give me another pair. The light scattering is just obnoxious, though. It doesn't actually prevent me from using the device. The voice recognition is about as good as one would expect (which is to say, borderline okay when one speaks slowly and deliberately, but pretty terrible with casual speech); no surprises there. It comes with a nice clear plastic lens insert, which will be good to protect the user's eyes in potentially messy situations. It's more lightweight than I expected, and the interface is pleasantly intuitive.

But the really exciting thing is that it seems to be caption-ready pretty much out of the box. I just started a hangout with myself, using my personal Gmail account on Glass and my professional Gmail account on my computer. My computer got video from Glass's camera (which was pointing over the top of my computer monitor, into my apartment's foyer), and Glass got video from my laptop's camera, which showed my own face wearing the admittedly dorky-looking Glass. I muted my laptop's microphone to test the sound quality of Glass's microphone and was impressed with its clarity, though of course we'll have to see how that alters depending on background noise and how far away the people we're captioning stand from the person wearing Glass. Best of all, though, when I typed into the hangout's chat window, the text came up instantly on Glass, with perfect clarity. So even though I haven't actually tested it with my steno machine yet, I think that as long as I use Plover or Eclipse with the Keyboard Macro setting turned on, I'll be able to send captions to Glass without having to commission any additional software. The Wi-Fi in the place I'll be using it is fairly reliable, but if it isn't I can always use my 4G hotspot as a backup. And if the microphone proves to be as good as it seems to be at first glance, I'll be able to caption remotely instead of having to stand next to my client, cramming myself into tiny spaces and generally making a nuisance of myself. The only downside is that I'll have to press "Enter" on my steno machine (which I've mapped to R-R, because it uses the two strongest fingers of the hands) after everything I write. But that won't be so terrible. I actually had to do that when I captioned a webinar two weeks ago, using Plover with the closed captioning feature built into InstantPresenter.com. It's a little tricky to get into the rhythm of pressing Enter each time, but it's certainly not a dealbreaker. More concerning is that Glass's display is designed to be above and to the right of a typical user's line of sight, forcing the user to glance upwards whenever they want to read anything on it, which might result in some eyestrain after constant use. I also haven't tested the battery life yet, though I'm hoping that Glass's battery will be able to withstand at least an hour or two of constant video chat. It's all very promising. Now I just have to get that optical defect sorted out, and then start testing it out with clients!

Our accessible cyberpunk future is so close, I can practically taste it.