fingerfans.dreamhosters.com

by **dr_leviathan** » 27 Jul 2005, 23:53

I discovered today that I can't buy a new Touch Stream LP. After panicking and looking all over for a retailer that still has one I was eventually turned on to this site.

Since I can't buy a new one I'm pondering the possibility of building one. It certainly isn't likely based on the amount of free time I've got, and I know that there are some keyboards still selling on EBay, but I'm still pondering the possibilities.

Does anyone have any understanding of the touch sensor technology used in the FingerWorks products? I would guess that it is some kind of capacitance array, but haven't researched it.

Is it the same as a touch-screen sensor? Or does it have higher resolution than a typical touchscreen?

It would seem that there are three parts to the product. (1) a touch sensor (2) software that interprets the touches and (3) a USB communication of standard mouse/keyboard operations.

I wouldn't want to have to build (1), but it doesn't seem impossible, especially if I can just buy touchscreen transducers. (2) seems doable and within my capabilities, dunno how to do (3) but I could learn or perhaps someone else already has experience with that.

If there was enough interest then perhaps we could put together an open-source hardware/software solution and submit an article to http://makemagazine.com/.

What do you think?[/url]

by **TorbenGB** » 28 Jul 2005, 13:33

Hi Doctor

First of all, welcome to the FingerFans community. You've come to the right place for sure, since FingerWorks (FW) is basically history.

I would advise you to keep an eye on eBay to find a TouchStream (TS). You should be able to pick one up for $500-700, but of course with bidding you never know in advance. Yes, that's more than the old retail price but that's a moot point since it's not available in retail anymore. It's a pressed market now and sadly the prices reflect this.

Building a TS from scratch would be a way-cool but immense project. Remember, the FW dudes spent years on getting from scratch to what they had in the end.
Figuring out how to decode the surface and turning it into a USB signal that your OS recognizes as keyboard/mouse devices -- that's probably the easy part. And making the software to tune and customize the device.
Making the sensor array itself is a different matter entirely. It's not simply a touch-screen overlay because such overlays can only handle one touch at a time -- just like a trackpad surface on any laptop computer. The FW sensor array can detect many touches at once, and that's what makes the FW technology so fundamentally different from other touch surfaces.

I don't want to discourage you in your first post, but building this as a home project doesn't sound very likely to succeed. If you measure up your costs in terms of time and materials, you'd probably be better off to spend the money on eBay after all. -- Or wait and see what the next spin of the globe reveals. We still don't know who bought FW, but speculations go in the direction that if it's Apple or similar, then we should start seeing some pretty cool (we hope) devices some time soon (six months to two years from now).

by **dr_leviathan** » 28 Jul 2005, 17:38

Good point that touch screens can only handle one touch.;

The touch sensor technology is an interesting puzzle. I've already come up with several ideas, but don't know which would be the lowest power.

When you have high-impedance transducers it is pretty easy to detect contact with something like a finger. The effort of many instrumentation applications is to reduce such effects rather than amplify them. The TS does a nice job of discriminating actual touch from mere proximity so that provides some clues about how it works. It also has nice resolution, so the mousing movement is sensitive enough that it actually works. That level of resolution might be tough to reproduce. Since it is fun to think about I'll ponder it some more, but I was curious is anyone had any insight.

Incidentally, I like the TS for its light touch and finger rest position. I could manage without the mousing gestures, and would actually prefer a keyboard that had more tactile feedback on the real location of the keys. So another idea I'm kicking around is that I could make something a bit in-between a TS and a regular keyboard, making the keys distinct unmovable plates embedded in a smooth surface such that I at least get the same typing effect and some of the gestures (shift, CTRL, etc).

by **frenchghost** » 28 Jul 2005, 23:35

Here is your road map:
Wayne Westerman's PhD Dissertation, Hand Tracking, Finger Identification, and Chordic Manipulation on a Multi-Touch Surface.
You can download a PDF and a PS version from this document here:
http://www.ee.udel.edu/~westerma/

Yes, I've read it and this is definitely where you get to know that this device is really an achievement :!:

-ivan

by **dr_leviathan** » 29 Jul 2005, 01:09

Yes, thank you. That is what I was looking for. Very interesting.

by **dr_leviathan** » 29 Jul 2005, 03:52

Yes. It is an array of high impedance open-end capacitors each connected to a very low bias-current amplifier. The plastic film is probably semi-conductive and the touch of the fingers provides the pico-amps necessary to give a nice healthy signal at the amplifier.

Not hard to do in a simple version, but probably tricky to get the resolution high and optimize the sensitivity/power/speed of the whole gadget.

For high impedance measurements the distance from the transducer (measuring element) to the amplifier has to be very short, so the plastic film is probably painted on one side with discrete metal plates (the shape of parallelograms according to the dissertation) that press-fit to a circuit board with surface mount (or smaller!) amplifiers directly on the other side. Even the initial prototype probably had such a configuration.

The shear number of amplifiers necessary is probably the main reason the keyboard was was so expensive. Even though they would be using custom IC's with multiple amplifiers per chip they still had a large number of channels to measure, and they had to read them all fast enough to detect rapid taps of the fingers so they had plenty of analog to digital converters (ADC's) feeding data to the firmware. A one-off homemade prototype would probably be very costly.

They used stretched parallelograms to reduce the resolution along one axis which helps keep the number of sensors lower than would be necessary for a homogeneous grid.

by **ivanw** » 29 Jul 2005, 12:36

More feeds for your quest:

BBC 2002 article
(with a link to a Windows Media report at bottom of page)

by **dr_leviathan** » 29 Jul 2005, 17:02

Thanks for the link ivanw.

I had a few ideas on the way home yesterday. The first one is obvious after reading that article where an early prototype keyboard had a paper surface.

(1) There are no metal painted on the backside of the TS plastic -- they're just on the printed circuit board (PCB) and the plastic is just a cover. The PCB part of the gadget is rather simple to manufacture. You can lay out a PCB design, send the files off to a manufacturer, and they'll mail you back the boards (yeah, so it is a few hundred dollars just for a few prints, but the cost per board goes way down for volume).

(2) You don't need amplifier's and ADC's for each sensor. A grid of smaller sensors, each going to a low-bias current digital input (pulled high with a high impedance (~100K Ohms) resistor) would work. In fact, those would be cheap enough today (and the data aquisition from all of them fast enough) that you probably wouldn't need the stretched parallelogram shaped sensors in the early prototypes -- a regular homogeneous grid of sensors would be simpler and not very expensive.

(3) A digital array would be cheap, but couldn't necessarily distinguish strong touch from light touch. I'm not sure it would be necessary to measure the strength of the touch at each point -- it seems that a high enough density of sensors with history information from previous measurements would be enough to distinguish a strong touch from a light one.

(3.1) If the strength of the touch must be measured then it might be possible to interleave high impedance inputs with others of slightly lower impedance. While this would appear to give only provide three states of information at any approximate point ( 00, 01, 11 ), the relative location of the two bits give you the gradient of the pressure distribution, which would be useful for modelling the strength of the touch.

(3.2) The strength of the touch can also be measured using strain gauges at flex points elsewhere in the keyboard. Basically, each tap would show up as a pulse in the strain measurement and the strength of each tap would be easy to detect. When strain measurements are correlated with a digital map the intent of the keystrokes and gestures would be unambiguously measured. This idea is actually intriguing...

Sorry if I'm boring anyone with my speculations. I'm just having fun thinking about the puzzle of building a similar keyboard. Next week I'll try to come up with an estimate of the material costs for building a TS prototype. Pondering how the TS is built has given me a few other ideas on another haptic gadget idea that is almost related to my day job.[/url]

by **Rqyteqto** » 29 Jul 2005, 17:51

What about the use of various piezo-electro materials that produce a variable current based on pressure?

And please continue boring us, well, at least me.

I personally feel this sort of input device, whether the exact mode used by Fingerworks or some other means, is fantastic. I am amazed they could go out of business. While I know all the rampant theories about conspiracy to bury various technologies, they really don't work out. If its that good, whoever bought it would want to use it, not bury it.

I have to assume the Fingerworks folks, while technically brilliant, were abysmal business people. Its often the case. There were three major reasons why Fingerworks failed, one - virtually zero exposure, two - very high price, expecially when compared to other input devices, and three - very steep learning curve. Of course, very few people ever found out about the steep learning curve because of the first two issues.

I literally stumbled upon Fingerworks while looking for alternative input devices. I was primarily interested in something like the twidler, something I could use while moving about, on the phone. I was unable to find any reviews on the products and so all I had to go on was simply Fingerworks web site which isn't the best.

The very high price also dampened my enthusiasm, I mean, $300 for an input device, its gotta be some kind of special inout device. Relying specifically on victims of RSI for a base of sales isn't a very good strategy. First off, they are generally hurting for money as they can't work and they already have significant medical expenses (plus no endorsements from or deals with rehab/insurance/government agency fir RSI). The web site is rather vague about any other advantages, though they are numerous.

So, anybody look into licensing the technology? Or how about reverse engineering a board. I'm not about to donate mine (I simply can't afford to) but perhaps somebody would be willing.

In any case, I'd love to see Multi-Touch return to the retail market.

by **dr_leviathan** » 29 Jul 2005, 19:58

I haven't done any work with piezo-electro materials, but as I think they measure pressure transitions rather than pressure. That is, they detect vibrations and sudden squeezes, but if you compress them with a steady force the signal vanishes. So for sounds and taps they might work, but they wouldn't work well for stuff like SHIFT gestures and soft mouse movements. But I may be wrong, perhaps someone will correct me.

I sounds to me like the FW people are still working on the technology, but whoever bought them out shut down the manufacturing of the existing products. So the technology is still promising and there are still people who are excited by its possibilities.

IANAL but as I understand it you can build a gadget that uses someone elses patent as long as you are not selling that gadget for profit. In order to sell a TS replacement you would have to license the technology. If FW was profitable when they were purchased then perhaps the buyers were not interested in having competitors in the market.

by **dr_leviathan** » 29 Jul 2005, 22:12

You may be correct lostlogic. The software that interprets the gestures is probably pretty complicated.

I haven't bothered speculating about the software because writing software is actually my primary strength and the project just seemed "doable" to me. I'm probably underestimating the work necessary for a couple of reasons:

(1) I only use a small subset of the TS feature set. I type, mouse, mouse-click, SHIFT, scroll and somtimes use four-finger CTRL and ALT -- that's about it (I'm unfamiliar with the "english spelling model" you mentioned, but I'm aware that TS's are cusomizable and there is some fancy API for building special gesture modes). In fact, I don't like the finger-drift correction stuff and was thinking I'd throw that out. A reduced feature set wouldn't be very hard to implement.

(2) I'm ignoring the blood sweat and tears involved in shoe-horning the software into an embedded processor that sits in the keyboard.

Jn any case, an effort to write the software would be pointless unless the hardware could be built, which makes the hardware question the first issue.

by **Rqyteqto** » 30 Jul 2005, 03:55

Whatever, we should all be alert and aware of what's going on. I, for one, can't imagine going back to normal input.

Like I said before, I don't subscribe to the quashing theories, mostly because there really isn't anything else out there similar to TouchStream and certainly nothing better for it to compete with. So if some party were to buy it, it would only make sense to use it. Even if that party had a lot invested in some other technology, it would make a lot more business sense to adapt whatever they are doing to TS and simply bury it. I am convinced Fingerworks was just poorly run. Heck, they did TS as a side line, most of their business was doing specialty board manufacture or something like that.

In any case, if you read the license, it prohibits any attempt to reverse engineer, copy, imitate or simulate the software and hardware and patents do prohibit any making or use, whether for profit or not. However, I do note the literature I have indicates patents pending so perhaps they never received an actual patent. I believe someone did research that so it probably a dead end. On the other hand, a design patent is only good for 14 years and they've been manufacturing TS products for at least 8 years.

I sure hope someone picks this up and runs with it. It would be a damn shame to let it just languish and die.

by **Rqyteqto** » 30 Jul 2005, 05:14

Here's something else I came across, similar and they have added pressure to the mix as well as multiple parties. Pretty interesting. They are also the people providing the basic technology for the Tactex Lemur which is the second link.

http://www.yourmarket.ca/sites/truenort ... 9142EAF1D7
www.thinkmig.com

Check it out and let me know what you think.

Here's another from the same root, Diamond-Touch.
http://www.index2005.dk/Members/kunisonisu/workObject

by **ivanw** » 30 Jul 2005, 11:33

Rqyteqto wrote:Here's another from the same root, Diamond-Touch.
http://www.index2005.dk/Members/kunisonisu/workObject

Yeah, they did it :!:

Download VIDEO (Quicktime)

by **Rqyteqto** » 31 Jul 2005, 01:55

I wonder is Mitsubishi isn't the large electronics concern that bought out Fingerworks. This DiamondTouch is very similar and for all intents and purposes a johnny-come-lately to the multi-touch arena. It could be they felt Fingerworks patents were broad enough to prevent their own and from what I can tell, seems to offer some advantages to theirs.

As best as I can make out, they use a capacitance system tied to connections in each persons' chair or station (in order to distinguish who's who). From the video, its quite amazing what its capable of.

I've written one of the head researchers in regards to if this is available, what costs are likely to be and when, etc.

by **dr_leviathan** » 01 Aug 2005, 18:37

I did some research into electrical components that might be used to build a TSclone and then came up with an estimate of what they would cost. This is a per/keyboard cost assuming that you're building about 10 keyboards in the end. I'mnot including the cost of tools such as solder iron, AVR microcontroller prototype board, etc, or personal labor costs. I've also ignored tax and shipping. This is all to just get an idea of the order of the magnitude:

Code: Select all: Part Description Qty Cost ---------------------------------------------------------------------------------- AD7908 8-bit, 8-channel ADC 100 $ 185 PCB 4-layer printed circuit board 2 78 Atmega 128 AVR microcontroller 1 36 misc electronics cmos, USB, capacitors, etc x 50 total $ 349

After thinking about it some more I decided that a digital sensor grid would just not work -- too many IC chips, too tight electronics -- the sensors would have to be analog. It turns out that many analog to digital converters (ADC) come with multiple channels and high-impance inputs, which makes a pre-amplifier unecessary.

The AD7908 has 8 input channels, an input impedance high enough (~3 MOhm, 20 pF) to detect the touch of a finger (~100 kOhm), and is rather cheap at about $1.85 each in quantities of 1000. I estimate that you could probably cover the keyboard with 100 ADC's, or about 8 sensors pads (1 ADC chip) per square inch. Notethat the sensor density is the biggest driver of the total cost. At 3V the AD7908 can run at very low power (~6 mW per chip) so I'm guessing the whole gadget would run at about 1 mW plus whatever the power usage of the microcontroller -- it won't get too hot.

It doesn't pay to try to go to 6-bit ADC's. They are all for high-speed applications and are much more expensive than slower 8-bit versions.

I didn't think it could be built on just a 2-layer PCB, but I'm guessing 4 layers would suffice.

by **-jeffB** » 02 Aug 2005, 14:25

People will do almost anything for fun, but if you have any life beyond this project, you'll find that you'd be further ahead working at McDonald's in the evenings until you've earned $1K to put toward a TouchStream on eBay. :-)

I do software and UI design, and my dissertation was on very low-level specification of user interfaces -- identifying user tasks based on mouse actions, figuring out exactly when you can tell exactly what task the user is trying to perform, and so forth. I feel pretty confident in saying that what Wayne accomplishes in this keyboard is really, really, REALLY hard.

Here are a couple of things we do know from comments Wayne and others made back when the "official" FingerWorks Forum was active:

The keyboard senses "pressure" by estimating contact patch area. I think it does that partly by measuring the actual capacitive response at a sensor, and partly by measuring the extent of a "patch" over more than one sensor. I think the keyboard surface can be considered rigid, in that space between your finger and the sensor doesn't change enough to affect capacitance levels.

The keyboard has an array of 16x80 sensors, divided into 16 rows and 40 columns on each side. You can see a map of this if you bring up MultiTouch Utilities, run the System Diagnostics, and fiddle with the keyboard while it's running.

The tracking and pointing algorithms rely on very heavy signal processing of the raw sensor data to smooth paths. Wayne said that they experimented with textured surfaces to make keys easier to distinguish, but it made it too hard to do smooth cursor movement.

The bottom line: I believe Wayne's work is considerably beyond what the typical PhD-level EE/software person could be expected to achieve, even working full-time on the problem. I have no doubt that you could recreate some of it, and maybe you're good enough to recreate it all and then improve it -- but if you are, there are probably a lot of other things you could work on that would be more fun and more beneficial to humanity. :-)

by **dr_leviathan** » 02 Aug 2005, 17:07

frenchghost posted a link to Wayne's PhD thesis up above which gives a good overview of the challenges involved in making a TS keyboard. Yes, it looks like a lot of work.

The sensors do indeed measure capacitance to detect proximity of the fingers, that is clear from the thesis. Yesterday while driving home I figured out how to measure changes in capacitance. The transducer is an RC circuit that is pulsed with a square wave. Some time after the rising edge of the wave the voltage is sampled. Under no influence from a nearby finger or palm the signal will have a known voltage at measurement time, however the close proximity of a finger will increase the capacitance enough to get a very different reading -- the measured voltage will be lower as capacitance increases.

It is interesting to know that the TS uses two 16x40 sensor grids. That is about 50% more than I estimated would be needed. Also, it means they were still using the stretched parallelogram system described in the thesis which reduces the number of sensors but makes data analysis tricky. I was thinking it would be easier to go with a hexagonal grid which would eliminate some of the complexities of the imaging algorithms, but increases the number of sensors required. A hexagonal grid over the same (half-keyboard) area with the same base resolution along the width would require 40x40 sensors (they pack a bit tighter in the other direction), however it may be that the shear increase in sensor count would allow you to get by with something closer to 30x30... which would still mean that the device would cost at least $580!

Yes, to get an interface to "kinda work" is much easeir than getting it to "work very well". I can imagine that tuning the mouse response took some effort and cleverness.

by **Rqyteqto** » 02 Aug 2005, 18:34

I got a reply from the MERL (Mitsubishi Electronics Research Laboratories) regarding DiamondTouch. They have shipped about 100 units to various institutions and companies for exploration, development and actual use. Cost is currently about $10K. They expect to release it commecially by the end of the year.

I still have a very strong suspicion this is what bought up FingerWorks. Mitsubishi is definitely a large electronics concern with lots of fingers in lots of pies. This DiamondTouch is strikingly similar to MultiTouch, the big differences being this uses a front project image (which could easily be added to MultiTouch, just need a projector and software to integrate the image with the touch surface and its multiple user capability. That's accomplished by using varying capacitance/resistance/inductance to create up to 4 I/O channels.

The impending release coupled with the overlap of technologies really makes this a strong candidate.

Dr. - if your estimates are even halfass reliable, then its a wonder FW could sell the units at the prices they did. Perhaps one reason for closing is they were subsidizing each board to a significant degree, hoping it would catch on and eventually pay out. It may be they were hoping for a breakthrough in costs with larger production numbers or some process to develop and come online. Perhaps they simply ran out of money and/or patience.

I guess time will tell. Either some future incarnation will appear at some higher, similar or lesser cost or it won't. Hopefully that will be sooner than later and the cost will be lesser than greater. In the meantime, I suggest anybody that has a working TS to take very good care if it.

by **dr_leviathan** » 02 Aug 2005, 19:03

Rqyteqto, the estimates are for making a batch of 10 keyboards, and only for the components (not labor or taxes). Of course, prices go way down when buying components in bulk, and even further down when custom IC chips are manufactured for a specific gadget.

I was trying to keep the estimate in the realm of a home-made version since a large production for the purpose of profit would run into legal problems, or would have to license the patented technology. A large scale operation would also require significant investment, marketing, and a profitable business model, which is beyond the scope of my ramblings.

I was curious to get an idea of what it would take to make a home-brew TS keyboard. My flawed answer, it seems, is a lot more than the TS originally sold for, but comparable to the auction price on Ebay.

by **-jeffB** » 02 Aug 2005, 19:54

Rqyteqto wrote:I got a reply from the MERL (Mitsubishi Electronics Research Laboratories) regarding DiamondTouch. They have shipped about 100 units to various institutions and companies for exploration, development and actual use. Cost is currently about $10K. They expect to release it commecially by the end of the year.

I still have a very strong suspicion this is what bought up FingerWorks. Mitsubishi is definitely a large electronics concern with lots of fingers in lots of pies. This DiamondTouch is strikingly similar to MultiTouch, the big differences being this uses a front project image (which could easily be added to MultiTouch, just need a projector and software to integrate the image with the touch surface and its multiple user capability. That's accomplished by using varying capacitance/resistance/inductance to create up to 4 I/O channels.

The impending release coupled with the overlap of technologies really makes this a strong candidate.

Ugh.

I just now Googled DiamondTouch, and I'm surprised I didn't stumble across this earlier. I've fallen behind on the literature -- I used to gobble up UIST proceedings as soon as they appeared. It looks like a really cool extension of a lot of the basic ideas of FingerWorks.

This makes me very, very afraid that Mitsubishi may have gobbled up FingerWorks just to get hold of patents that might otherwise have blocked them. It's quite possible that, even if FingerWorks had their patents first, Mitsubishi had enough legal muscle to run FingerWorks into the ground (assuming they hadn't already crashed based on their business model). They may have approached Wayne with an offer he couldn't refuse: "Defend your patents against us in court, or sell all the rights to us for this tidy sum."

If FingerWorks already was on the verge of financial collapse -- and the problems with customer service, along with their high costs and apparently low sales, make this seem likely -- I can see this kind of thing overcoming Wayne's probable objections to shuttering the company and taking the technology out of circulation. It's an ugly scenario, but based on the startups that I've ridden into the ground, it seems very, very plausible.

But I sure hope it isn't true.

by **Rqyteqto** » 03 Aug 2005, 04:55

jeff - I agree to some extent. However, if it was just a licensing issue, Mitsubishi probably could have simply licensed the FingerWorks technology and moved on down the line. Then again, its possible they saw the various FingerWorks products as direct and substantially lower cost competition to DiamondTouch. I don't know.

One thing is for sure, if you have a TS or any FingerWorks product, you have something you should hang onto. It may well never be seen again. I wish I'd had the foresight to buy a half dozen when I could.

fingerfans.dreamhosters.com

touch sensor technology

touch sensor technology

Ideas

DiamondTouch

Re: DiamondTouch

Who is online