All Over The Map

In 1998/9, several of us in Sun Labs and with help from Dallas Semiconductor, developed a bunch of technology for monitoring and securely  controlling appliances for home automation applications. Time marches on but I can;t get over how many things in this article are ideas that Sun’s forward thinking CEO Scott McNealy presented to audiences all over the world many times a long time ago. I can’t find it but there were some press rants similar to Cramer’s rants about how stupid the ideas were. It looks like others are coming up with applications. We developed and built working prototypes of the following. If I were starting again, I would probably start with the Sun SPOT technology from Sun Labs.

• Double door refrigerator interfaced to the Internet
• Advanced Oven(Microwave, Infrared and Convection)
• A general purpose X10 device for interfacing to various manufacturer protocols using a PIC. ( will look for a picture of one)
• A light switch that was addressable using a tini board. (Dallas Semiconductor TINI they did all the heavy lifting, we played a minor role, tech note)
• We had prototypes for putting Java Card technology on each device to implement secure challenge response and point to point encryption with secret keys
• Dallas provided weather stations and low cost sensors that we wanted to use for telemetry. (Cheap wireless was not available, it still isn’t looking for 2-3 dollars for wireless)

When the Internet bust happened we got lumped into “questionable” research efforts and they were probably right. We could not show the cost benefit and the utility. There really are tremendous safety issues. Be careful trusting someone with this kind of technology, they could burn your house down in one scenario.  A few years later I looked into it some more from the aspect of monitoring energy usage.  I blogged about it a lot later than when I had first worked on it. I was just rereading the code from 2000 and I see it is very similar to the JSON idea. We were not smart enough to come up with JSON but we wanted machines  to talk to web servers and get pages, in 1998/1999. So we had the server return java property type files in both this project and an earlier one called Java Wallet. Necessity is the mother of invention. I discussed some of this in Simultaneity of Invention.

A friend told me about PowerMeter from Google. I hope that it takes off and eventually grows into a home you can control from the Internet.  It does start to feel like “Matrix” and or “SkyNet”  when “it” can get to a light switch in your home.  I wish we had started the project ten years later.

References:

• Some work on http addressable switches, http://www.nt.ntnu.no/users/skoge/prost/proceedings/ifac2002/data/content/01783/1783.pdf

• Some dated material that with a little work could be turned into a REST interface. https://research.sun.com/brazil/api/tutorial/html/index.html
• Brazil Naws as a framework for these types of applications. http:www.skynet.ie/~dejavu/oldsite/final.report.pdf
• More detail on TINI http://www.javaworld.com/javaworld/jw-11-1999/jw-11-tini.html?page=1

• “Java in embedded systems,” Rinaldo Di Giorgio (JavaWorld, September 1996)
  http://www.javaworld.com/javaworld/jw-09-1996/jw-09-javadev.html
• “An introduction to the URL programming interface,” Rinaldo Di Giorgio (JavaWorld, September 1999)
  http://www.javaworld.com/javaworld/jw-09-1999/jw-09-javadev.html
• “An instrumentation network for weather data on the Web,” Rinaldo Di Giorgio (JavaWorld, May 1999)
  http://www.javaworld.com/javaworld/jw-05-1999/jw-05-javadev.html
• “A conversation with Bill Joy” (Sun Journal, n.d.)
  http://www.sun.com/SunJournal/v2n3/EmergingTech.html
• An explanation of using Java technology to standardize realtime development
  http://java.sun.com/products/embeddedjava/real-time.html
• The community resource for Jini technology
  http://www.jini.org

One of the most enjoyable environments I worked in, was Sun Labs around 1996-2000. During that short period of time I worked on a few projects that are still not ubiquitous or routinely available, JavaPanel ( carry compute with you and plug into a flat panel), Screen Scraping, Tini boards/iButtons and Payment Authentication/Devices (phones that are credit cards, plastic credit cards yuk), smart appliances/homes (mapping you entire house into a url address space). Some of the projects however did become mainstream and had the “The Simultaneity of Invention” property. The two projects I want to discuss are the URL Programming Interface (UPI) and in another post, the Unstructured Data Extraction supported by  the Brazil/Naws advanced web server. UPI was very similar to REST. At the time we were not aware of the work going on in REST.  If is not even clear to me that this is an invention but rather an application, invention doesn’t map well to software for me, I prefer new application, or simplification over invention.  We were more interested in using UPI so we did little to document or describe it. Roy Feilding, gave the topic the careful thought and analysis it required in http://www.ics.uci.edu/~fielding/pubs/dissertation/top.htm. We built and wrote alot of code interfacing devices, mounting web sites into one address space for a year or two. It just made sense to map things like devices, smartcards, phone numbers, refrigerators, weather stations and meeting rooms and people, etc. into urls.  A surviving presentation can be found here.  An article in JavaWorld from  1999,  describes what we had been doing for  a year or two prior to the article.  Someone took the time to explain what it was, in this paper, upi-eng.  A short quote from R. L. Heilbroner found in “Does Technology Drive History?”

The phenomenon of simultaneous discovery is well known[1]. From our view, it argues that the process of discovery takes place along a well-defined frontier of knowledge rather than in grab-bag fashion. Admittedly the concept of “simultaneity” is “impressionistic”[2],  but the related phenomenon of technological “clustering” again suggests that technical evolution follows a sequential and determinate rather than random course[3].

References

[1] Robert K. Metorn, “Singletons and Mutiples in Scientific Discovery”

[2] John Jewkes, David Sawers and Richard Stillerman, The Sources of Invention

[3] One can count 21 basically different means of flying.  Current Sociology 1 (1952)

Web based applications running in a browser often require persistent storage.  There are many ways one can achieve persistence in a browser based application. Web Dav provides several advantages.

1. An easy to understand file system based model
2. Was designed to work with http
3. Wide spread client support
4. Easy to debug you can browse the file store using a client

Ajax based applications have a security restriction requiring that you post back to the same server domain as the page you are currently on.  Consider the following use case.

Your storage is at www.eapps.com  and your users are all over the Internet, using a widget you have made available for them to upload dynamically generated RDF files.  Unless a proxy is provided the widget will experience a cross domain issue. You can easily see cross domain issues using Firebug.

We have put together a development kit that allows you to develop Javascript based Web Dav applications with cross domain support. The approach we took is to create a proxy handler that maps the remote Web Dav location into the same origin as your page.  We used the following components.

• davclient.js
• naws

You will be able to download or contribute to  the naws version from kenai here

You will be able to download or contribute to the Glassfish version from kenai here

If you are interested in participating,  there are two forum discussions, one for naws and one for naws/glassfish.

I spent two days on the following problem and I thought I would write about it in case someone else figures out how to do it. I can’t invest more time in the convoluted class structure of HttpURLConnection. The class cohesion and inheritance properties are beyond my level of understanding.  I have a simple need, I want to post WebDAV requests to a server from a Java application. When I try to set the request methods for HTTPURLConnection, I get an error for things like MKCOL and PROPFIND. Ok I think no problem I will just extend, the class. I try that and can’t get past various issues. I am sure the Security argument will be used as a justification. In fact the source code forHTTPUrlConnection has a comment to that effect. The comment is very odd, I can just create a PlainConnect and post.

1. You need to supply a handler, you can’t use the default handler
2. It appears all the https support doesn’t work when I try it

I don’t understand why this has to be so hard, one of the main features of Object oriented programming was to allow code to be extended and reused. Instead I have to resort to grabbing the plain socket and then proceeding to generate http requests like PROPFIND directly. Maybe HTTPURLConnection has a long  past and because there is such legacy it will never be fixed to support the present. y original intent was to simply override a method and add some parameters, 16 hours of investment later, I had to use an alternate implementation of HTTPUrlConnection. I realize the need for stability and backward compatibility, so a simple solution is to provide other implementations and deal with the class bloat.  This issue reminds me of well documented research on bug fixes for OS/360  where you could no longer make a change because of the long tail and side effects.  Another option I had was to just switch to Ruby, there is no issue with using other HTTP methods with Ruby, but I would rather see Java through this bump in the road. I have attached some of the Java code I could not get to work. A future post will provide a naws implementation that allows you to use Web DAV directly from Javascript without a domain issue, that is you can run it against any Web DAV service. The class below is based on the following code segment, from a website that looks like they took it from the original IBM WebDav implementation (DAV4J) that is no longer open source.

package contrib;

import java.io.*;
import java.net.*;

public class DavUrlConnection extends sun.net.www.protocol.http.HttpURLConnection {

    private static final String[] methods = {"GET", "POST", "HEAD", "OPTIONS", "PUT", "DELETE", // from HTTP/1.1
        "MKCOL", "COPY", "MOVE", "PROPFIND", "PROPPATCH", "LOCK", "UNLOCK", "SEARCH"};

    protected DavUrlConnection(URL u, Handler handler) throws IOException {
        super(u, handler);
    }

    public DavUrlConnection(URL u, String host, int port) throws IOException {
        super(u, host, port);
    }

    public synchronized InputStream getInputStream() throws IOException {
        InputStream is = null;
        try {
            is = super.getInputStream();
        } catch (IOException exc) {
        }
        return is;
    }

    public synchronized OutputStream getOutputStream() throws IOException {
        OutputStream os = null;
        String savedMethod = method;
        // see if the method supports output
        if (method.equals("GET") || method.equals("PUT") || method.equals("POST") ||
                method.equals("PROPFIND") || method.equals("PROPPATCH") ||
                method.equals("MKCOL") || method.equals("MOVE") || method.equals("COPY") ||
                method.equals("LOCK")) {
            // fake the method so the superclass method sets its instance variables
            method = "PUT";
        } else {
            // use any method that doesn't support output, an exception will be
            // raised by the superclass
            method = "DELETE";
        }
        os = super.getOutputStream();
        method = savedMethod;
        return os;
    }

    public void setRequestMethod(String method) throws ProtocolException {
        if (connected) {
            throw new ProtocolException("Cannot reset method once connected");
        }
        // prevent clients from specifying invalid methods. This prevents experimenting
        // with new methods without editing this code, but should be included for
        // security reasons.
        for (int i = 0; i < methods.length; i++) {
            if (methods[i].equals(method)) {
                this.method = method;
                return;
            }
        }
        throw new ProtocolException("Invalid WebDAV method: " + method);
    }
}