I am currently a Senior Consultant at Amazon Web Services (AWS) Professional Services, specializing in Big Data and Data Science. I help AWS customers solve their business problems by adopting AWS technologies, fitting the technology choices and implementation to the organization, the task, the issues and associated risks.
Recent publications include:
My IT interests include adoption and applications of emerging technologies; big data, information management, data access and discovery; spatio-temporal data and databases; integration of biological, physical, spatio-temporal and business data; and scientific data.
I'm currently particularly interested in understanding the risks involved in Machine Learning projects; in identifying methods to identify, assess, manage and mitigate the risks. I believe some of the public ML failures are the result of risks inherent in the technology or implementation, that were perhaps known by the implementation team but not visible to or understood by their management. In some cases, methods exist that can be adapted to avoid these occurrences. In other cases, new methods must be developed. What fun!
From July 2014 to August 2015 I was a post-doctoral fellow in Computer Science at the Maseeh College of Engineering & Computer Science, Portland State University, in Portland Oregon. Just prior, from Fall 2009 to August 2015, I was a PhD candidate, working with Professor Dave Maier at Portland State University, and with the Center for Margin Observation and Prediction (CMOP) (part of OHSU).
My prior industry position was Executive IT Architect at IBM, which I left to pursue my Masters and PhD. Here are links for: a) a summary of recent accomplishments and b) a list of my industry publications.
My 15 minutes of fame (well, it's well over 30 minutes by now) are documented in many places, including: Tolkien Gateway (which references a parody of the game I didn't know about), World of Spectrum (with a picture I have no memory of), and a Wikipedia entry (none of which I wrote). There's even a walk-through, here.
As of "the day I last checked", "The Hobbit (1982)" was the second most downloaded item in the Internet Archive's Historical Software Collection, where you can 'play it again' in an emulator. And someone spent far longer than it took Phil and me to code it on reverse-engineering it (from the bytecode, no less; the original was in Z80 assembler and as far as I know the source code was never released) and exposing the innards (Wilderland).
I hereby apologize for making Thorin spend so much of his time singing about gold... Too short a character action list, you see...
A random, unscientific sampling of the (extensive) media coverage includes:
... and I even wrote a recent blog article for the Australian Center for the Moving Image, Ruminations on The Hobbit Fandom. Here is their profile of me. There's also a paper I presented at the 'Born Digital and Cultural Heritage Conference', ACMI, Melbourne Australia, in June 2014.
I received my PhD in June, 2014, from Portland State University in Oregon. My dissertation topic is "Ranked Similarity Search of Scientific Datasets: An Information Retrieval Approach". My thesis committee consisted of:
My dissertation work was a system I called "Data Near Here".
"Data Near Here" applies ideas from the field of cognitive science and spatial cognition, Information Retrieval and Internet search to massive archives of scientific datasets. I address the following problem: with the explosion of data collected by scientists and stored in many files, many formats, many naming conventions, how do scientists find data that matches their research needs?
I use a running example of a scientist searching for salinity observations collected in of May 2009, near the Astoria-Megler bridge. A screenshot of DNH running over CMOP's archive can be seen below. Note that in this case, there are no exact matches for the scientist's search terms as formulated; given no exact matches, the tool presents an ordered list with the "closest" matches at the top.
Similar in concept to the way an Internet text search engine operates, I focus on providing a set of results ranked by similarity to a scientist's search; however, rather than text webpages, my users are searching for scientific (primarily numeric) data. I assume that after reviewing the search results, the scientist will wish to download, visualize or otherwise process selected datasets using other tools. Thus, the search engine is complementary to existing analysis and visualization technologies.
In later work, we started testing these concepts on genomic data, in a system we called "Data Like This".
A set of crawlers scan an archive of datasets, asynchronously. I create a brief summary of the contents of each dataset, modeled on the internal mental model scientists have of their data, and store the summary in a metadata catalog using a simple, consistent abstraction. The current prototype handles several different file types, and the scanning process can be easily extended to handle additional file types and formats.
The user enters search criteria into a UI. (Note: "I am not a UI designer, and this is not the topic of my research.") A search engine searches over the metadata and returns ranked search results of the "closest matches" to the query, in real-time. Searches can include location, time, variable names of interest, or desired ranges for the data values. The results are displayed in a list (and, if geolocation information is available, on a map), along with brief summary information. The results can be downloaded for analysis or plotted in linked data analysis or visualization tools. A link leads to a page that shows the full metadata available for that dataset, thus providing the scientist with additional information upon which to make analysis decisions, if desired.
"Data Near Here" is described in the following publications:
... and, of course, at great length in my dissertation: "Ranked Similarity Search of Scientific Datasets: An Information Retrieval Approach" (of which sections, surprisingly, strongly resemble the above listed papers. But there is additional content there, too). [local copy]
A patent "A Search Tool that Utilizes Numerical Scientific Metadata Matched Against User-Entered Parameters Edit", United States Patent US8560531 B2 was issued on issued October 15, 2013 (filed July 1, 2011). Inventors: Veronika Megler, David Maier; Joint IBM/Portland State University.
Data Near Here is in production at CMOP, for use by registered users only. It will be opened to outside users in the (hopefully near) future (reasons for the delay can be found here). The CMOP production implementation currently focuses primarily on CMOP's own data archive. Data from other archives was to be searchable via this implementation in the future, and development instances of the system provided that capability.
We also explored application of the same ideas and concepts to genomics data ("Data Like This"), and developed an early proof-of-concept.
I also contributed to the Portland Observatory project. This research project intended to explore how one might architect and build an observatory that understands and adapts to the wide variety of data gathered or otherwise available in a single domain. Our local city of Portland, Oregon, was a laboratory and example within which to explore these concepts.
A second use case is described in Improving Data Quality in Intelligent Transportation Systems, Tech Report S23204, August 2015 (Megler, Tufte & Maier).
I am involved in other "random" (fun) research-related activities. They include:
I originally fantasized about peaceful nature sounds playing until someone asked me a question, whereupon the system would play a short tape to the other participants of me saying, "I'm sorry, I was talking to the mute button", while showing me a transcribed version of immediately preceding context so I could formulate my response. My co-author came up with the excellent title and acronym. (Names in alphabetical order, by IBM standard. Getting them to keep the title was tough.)