touch sensor technology

[dr_leviathan]

I discovered today that I can't buy a new Touch Stream LP. After panicking and looking all over for a retailer that still has one I was eventually turned on to this site.

Since I can't buy a new one I'm pondering the possibility of building one. It certainly isn't likely based on the amount of free time I've got, and I know that there are some keyboards still selling on EBay, but I'm still pondering the possibilities.

Does anyone have any understanding of the touch sensor technology used in the FingerWorks products? I would guess that it is some kind of capacitance array, but haven't researched it.

Is it the same as a touch-screen sensor? Or does it have higher resolution than a typical touchscreen?

It would seem that there are three parts to the product. (1) a touch sensor (2) software that interprets the touches and (3) a USB communication of standard mouse/keyboard operations.

I wouldn't want to have to build (1), but it doesn't seem impossible, especially if I can just buy touchscreen transducers. (2) seems doable and within my capabilities, dunno how to do (3) but I could learn or perhaps someone else already has experience with that.

If there was enough interest then perhaps we could put together an open-source hardware/software solution and submit an article to .

What do you think?[/url]

[TorbenGB]

Hi Doctor
First of all, welcome to the FingerFans community. You've come to the right place for sure, since FingerWorks (FW) is basically history.

I would advise you to keep an eye on eBay to find a TouchStream (TS). You should be able to pick one up for $500-700, but of course with bidding you never know in advance. Yes, that's more than the old retail price but that's a moot point since it's not available in retail anymore. It's a pressed market now and sadly the prices reflect this.

Building a TS from scratch would be a way-cool but immense project. Remember, the FW dudes spent years on getting from scratch to what they had in the end.
Figuring out how to decode the surface and turning it into a USB signal that your OS recognizes as keyboard/mouse devices -- that's probably the easy part. And making the software to tune and customize the device.
Making the sensor array itself is a different matter entirely. It's not simply a touch-screen overlay because such overlays can only handle one touch at a time -- just like a trackpad surface on any laptop computer. The FW sensor array can detect many touches at once, and that's what makes the FW technology so fundamentally different from other touch surfaces.

I don't want to discourage you in your first post, but building this as a home project doesn't sound very likely to succeed. If you measure up your costs in terms of time and materials, you'd probably be better off to spend the money on eBay after all. -- Or wait and see what the next spin of the globe reveals. We still don't know who bought FW, but speculations go in the direction that if it's Apple or similar, then we should start seeing some pretty cool (we hope) devices some time soon (six months to two years from now).

[dr_leviathan]

Good point that touch screens can only handle one touch.;

The touch sensor technology is an interesting puzzle. I've already come up with several ideas, but don't know which would be the lowest power.

When you have high-impedance transducers it is pretty easy to detect contact with something like a finger. The effort of many instrumentation applications is to reduce such effects rather than amplify them. The TS does a nice job of discriminating actual touch from mere proximity so that provides some clues about how it works. It also has nice resolution, so the mousing movement is sensitive enough that it actually works. That level of resolution might be tough to reproduce. Since it is fun to think about I'll ponder it some more, but I was curious is anyone had any insight.

Incidentally, I like the TS for its light touch and finger rest position. I could manage without the mousing gestures, and would actually prefer a keyboard that had more tactile feedback on the real location of the keys. So another idea I'm kicking around is that I could make something a bit in-between a TS and a regular keyboard, making the keys distinct unmovable plates embedded in a smooth surface such that I at least get the same typing effect and some of the gestures (shift, CTRL, etc).

[frenchghost]

[dr_leviathan]

[dr_leviathan]

[ivanw]

[dr_leviathan]

Thanks for the link ivanw.

I had a few ideas on the way home yesterday. The first one is obvious after reading that article where an early prototype keyboard had a paper surface.

(1) There are no metal painted on the backside of the TS plastic -- they're just on the printed circuit board (PCB) and the plastic is just a cover. The PCB part of the gadget is rather simple to manufacture. You can lay out a PCB design, send the files off to a manufacturer, and they'll mail you back the boards (yeah, so it is a few hundred dollars just for a few prints, but the cost per board goes way down for volume).

(2) You don't need amplifier's and ADC's for each sensor. A grid of smaller sensors, each going to a low-bias current digital input (pulled high with a high impedance (~100K Ohms) resistor) would work. In fact, those would be cheap enough today (and the data aquisition from all of them fast enough) that you probably wouldn't need the stretched parallelogram shaped sensors in the early prototypes -- a regular homogeneous grid of sensors would be simpler and not very expensive.

(3) A digital array would be cheap, but couldn't necessarily distinguish strong touch from light touch. I'm not sure it would be necessary to measure the strength of the touch at each point -- it seems that a high enough density of sensors with history information from previous measurements would be enough to distinguish a strong touch from a light one.

(3.1) If the strength of the touch must be measured then it might be possible to interleave high impedance inputs with others of slightly lower impedance. While this would appear to give only provide three states of information at any approximate point ( 00, 01, 11 ), the relative location of the two bits give you the gradient of the pressure distribution, which would be useful for modelling the strength of the touch.

(3.2) The strength of the touch can also be measured using strain gauges at flex points elsewhere in the keyboard. Basically, each tap would show up as a pulse in the strain measurement and the strength of each tap would be easy to detect. When strain measurements are correlated with a digital map the intent of the keystrokes and gestures would be unambiguously measured. This idea is actually intriguing...


Sorry if I'm boring anyone with my speculations. I'm just having fun thinking about the puzzle of building a similar keyboard. Next week I'll try to come up with an estimate of the material costs for building a TS prototype. Pondering how the TS is built has given me a few other ideas on another haptic gadget idea that is almost related to my day job.[/url]

[Rqyteqto]

What about the use of various piezo-electro materials that produce a variable current based on pressure?

And please continue boring us, well, at least me.

I personally feel this sort of input device, whether the exact mode used by Fingerworks or some other means, is fantastic. I am amazed they could go out of business. While I know all the rampant theories about conspiracy to bury various technologies, they really don't work out. If its that good, whoever bought it would want to use it, not bury it.

I have to assume the Fingerworks folks, while technically brilliant, were abysmal business people. Its often the case. There were three major reasons why Fingerworks failed, one - virtually zero exposure, two - very high price, expecially when compared to other input devices, and three - very steep learning curve. Of course, very few people ever found out about the steep learning curve because of the first two issues.

I literally stumbled upon Fingerworks while looking for alternative input devices. I was primarily interested in something like the twidler, something I could use while moving about, on the phone. I was unable to find any reviews on the products and so all I had to go on was simply Fingerworks web site which isn't the best.

The very high price also dampened my enthusiasm, I mean, $300 for an input device, its gotta be some kind of special inout device. Relying specifically on victims of RSI for a base of sales isn't a very good strategy. First off, they are generally hurting for money as they can't work and they already have significant medical expenses (plus no endorsements from or deals with rehab/insurance/government agency fir RSI). The web site is rather vague about any other advantages, though they are numerous.

So, anybody look into licensing the technology? Or how about reverse engineering a board. I'm not about to donate mine (I simply can't afford to) but perhaps somebody would be willing.

In any case, I'd love to see Multi-Touch return to the retail market.

[dr_leviathan]

I haven't done any work with piezo-electro materials, but as I think they measure pressure transitions rather than pressure. That is, they detect vibrations and sudden squeezes, but if you compress them with a steady force the signal vanishes. So for sounds and taps they might work, but they wouldn't work well for stuff like SHIFT gestures and soft mouse movements. But I may be wrong, perhaps someone will correct me.

I sounds to me like the FW people are still working on the technology, but whoever bought them out shut down the manufacturing of the existing products. So the technology is still promising and there are still people who are excited by its possibilities.

IANAL but as I understand it you can build a gadget that uses someone elses patent as long as you are not selling that gadget for profit. In order to sell a TS replacement you would have to license the technology. If FW was profitable when they were purchased then perhaps the buyers were not interested in having competitors in the market.

[dr_leviathan]

You may be correct lostlogic. The software that interprets the gestures is probably pretty complicated.

I haven't bothered speculating about the software because writing software is actually my primary strength and the project just seemed "doable" to me. I'm probably underestimating the work necessary for a couple of reasons:

(1) I only use a small subset of the TS feature set. I type, mouse, mouse-click, SHIFT, scroll and somtimes use four-finger CTRL and ALT -- that's about it (I'm unfamiliar with the "english spelling model" you mentioned, but I'm aware that TS's are cusomizable and there is some fancy API for building special gesture modes). In fact, I don't like the finger-drift correction stuff and was thinking I'd throw that out. A reduced feature set wouldn't be very hard to implement.

(2) I'm ignoring the blood sweat and tears involved in shoe-horning the software into an embedded processor that sits in the keyboard.

Jn any case, an effort to write the software would be pointless unless the hardware could be built, which makes the hardware question the first issue.

[Rqyteqto]

Whatever, we should all be alert and aware of what's going on. I, for one, can't imagine going back to normal input.

Like I said before, I don't subscribe to the quashing theories, mostly because there really isn't anything else out there similar to TouchStream and certainly nothing better for it to compete with. So if some party were to buy it, it would only make sense to use it. Even if that party had a lot invested in some other technology, it would make a lot more business sense to adapt whatever they are doing to TS and simply bury it. I am convinced Fingerworks was just poorly run. Heck, they did TS as a side line, most of their business was doing specialty board manufacture or something like that.

In any case, if you read the license, it prohibits any attempt to reverse engineer, copy, imitate or simulate the software and hardware and patents do prohibit any making or use, whether for profit or not. However, I do note the literature I have indicates patents pending so perhaps they never received an actual patent. I believe someone did research that so it probably a dead end. On the other hand, a design patent is only good for 14 years and they've been manufacturing TS products for at least 8 years.

I sure hope someone picks this up and runs with it. It would be a damn shame to let it just languish and die.

[Rqyteqto]

Here's something else I came across, similar and they have added pressure to the mix as well as multiple parties. Pretty interesting. They are also the people providing the basic technology for the Tactex Lemur which is the second link.

http://www.yourmarket.ca/sites/truenort ... 9142EAF1D7
www.thinkmig.com

Check it out and let me know what you think.

Here's another from the same root, Diamond-Touch.
http://www.index2005.dk/Members/kunisonisu/workObject

[ivanw]

[Rqyteqto]

[dr_leviathan]

[-jeffB]

People will do almost anything for fun, but if you have any life beyond this project, you'll find that you'd be further ahead working at McDonald's in the evenings until you've earned $1K to put toward a TouchStream on eBay. :-)

I do software and UI design, and my dissertation was on very low-level specification of user interfaces -- identifying user tasks based on mouse actions, figuring out exactly when you can tell exactly what task the user is trying to perform, and so forth. I feel pretty confident in saying that what Wayne accomplishes in this keyboard is really, really, REALLY hard.

Here are a couple of things we do know from comments Wayne and others made back when the "official" FingerWorks Forum was active:

The keyboard senses "pressure" by estimating contact patch area. I think it does that partly by measuring the actual capacitive response at a sensor, and partly by measuring the extent of a "patch" over more than one sensor. I think the keyboard surface can be considered rigid, in that space between your finger and the sensor doesn't change enough to affect capacitance levels.

The keyboard has an array of 16x80 sensors, divided into 16 rows and 40 columns on each side. You can see a map of this if you bring up MultiTouch Utilities, run the System Diagnostics, and fiddle with the keyboard while it's running.

The tracking and pointing algorithms rely on very heavy signal processing of the raw sensor data to smooth paths. Wayne said that they experimented with textured surfaces to make keys easier to distinguish, but it made it too hard to do smooth cursor movement.

The bottom line: I believe Wayne's work is considerably beyond what the typical PhD-level EE/software person could be expected to achieve, even working full-time on the problem. I have no doubt that you could recreate some of it, and maybe you're good enough to recreate it all and then improve it -- but if you are, there are probably a lot of other things you could work on that would be more fun and more beneficial to humanity. :-)

[dr_leviathan]

frenchghost posted a link to Wayne's PhD thesis up above which gives a good overview of the challenges involved in making a TS keyboard. Yes, it looks like a lot of work.

The sensors do indeed measure capacitance to detect proximity of the fingers, that is clear from the thesis. Yesterday while driving home I figured out how to measure changes in capacitance. The transducer is an that is pulsed with a square wave. Some time after the rising edge of the wave the voltage is sampled. Under no influence from a nearby finger or palm the signal will have a known voltage at measurement time, however the close proximity of a finger will increase the capacitance enough to get a very different reading -- the measured voltage will be lower as capacitance increases.

It is interesting to know that the TS uses two 16x40 sensor grids. That is about 50% more than I estimated would be needed. Also, it means they were still using the stretched parallelogram system described in the thesis which reduces the number of sensors but makes data analysis tricky. I was thinking it would be easier to go with a hexagonal grid which would eliminate some of the complexities of the imaging algorithms, but increases the number of sensors required. A hexagonal grid over the same (half-keyboard) area with the same base resolution along the width would require 40x40 sensors (they pack a bit tighter in the other direction), however it may be that the shear increase in sensor count would allow you to get by with something closer to 30x30... which would still mean that the device would cost at least $580!

Yes, to get an interface to "kinda work" is much easeir than getting it to "work very well". I can imagine that tuning the mouse response took some effort and cleverness.

[Rqyteqto]

I got a reply from the MERL (Mitsubishi Electronics Research Laboratories) regarding DiamondTouch. They have shipped about 100 units to various institutions and companies for exploration, development and actual use. Cost is currently about $10K. They expect to release it commecially by the end of the year.

I still have a very strong suspicion this is what bought up FingerWorks. Mitsubishi is definitely a large electronics concern with lots of fingers in lots of pies. This DiamondTouch is strikingly similar to MultiTouch, the big differences being this uses a front project image (which could easily be added to MultiTouch, just need a projector and software to integrate the image with the touch surface and its multiple user capability. That's accomplished by using varying capacitance/resistance/inductance to create up to 4 I/O channels.

The impending release coupled with the overlap of technologies really makes this a strong candidate.

Dr. - if your estimates are even halfass reliable, then its a wonder FW could sell the units at the prices they did. Perhaps one reason for closing is they were subsidizing each board to a significant degree, hoping it would catch on and eventually pay out. It may be they were hoping for a breakthrough in costs with larger production numbers or some process to develop and come online. Perhaps they simply ran out of money and/or patience.

I guess time will tell. Either some future incarnation will appear at some higher, similar or lesser cost or it won't. Hopefully that will be sooner than later and the cost will be lesser than greater. In the meantime, I suggest anybody that has a working TS to take very good care if it.

[dr_leviathan]

Rqyteqto, the estimates are for making a batch of 10 keyboards, and only for the components (not labor or taxes). Of course, prices go way down when buying components in bulk, and even further down when custom IC chips are manufactured for a specific gadget.

I was trying to keep the estimate in the realm of a home-made version since a large production for the purpose of profit would run into legal problems, or would have to license the patented technology. A large scale operation would also require significant investment, marketing, and a profitable business model, which is beyond the scope of my ramblings.

I was curious to get an idea of what it would take to make a home-brew TS keyboard. My flawed answer, it seems, is a lot more than the TS originally sold for, but comparable to the auction price on Ebay.

[-jeffB]

[Rqyteqto]