Geezerwicket

I have just watched a video that landed on my virtual desk from the TED Talks. TED ("Technology, Entertainment, Design") is an extraordinarily exciting group of thinkers/visioners/doers. They are artists, scientists, musicians, activists. It's hard to say what they are, actually. What I know is that they create this magic space where they get together and talk about ideas.

But they don't just talk. They also try to effect change. That makes them especially wonderful.

The Bad News
I had an invitation to attend a Ted Conference last year. But at $3500 or so it's a steep price tag for a girl with an head full of ideas, tons of passion but an empty portfolio and emptier pockets. Besides, I wonder, How can I contribute meaningfully to these discussions? What do I bring to the table that is unique to me? So while I wrestle with my demons, I squirrel my pennies away and work on my portfolio, ...

The Good News
The good news is that Ted means business. They want their ideas to be out there so they put videos of their talks online for the world to see. Free. See for yourself.

So, back to the reason for this post. in my inbox today was the Allosphere. Well really, it was announcement about TED tweets but really, do the mechanics of how I found it matter? The idea of an allosphere in my inbox is so much more romantic.

So what is the Allosphere. Well, at first glance it appears to be 1/3 Cerebro (X-Men), 1/3 Holodeck (Star Trek) and the rest is a whole lot of other magic thrown in. A less star-struck person might tell you, as Ted does, that the Allosphere is "an entirely new way to see and interpret scientific data, in full color and surround sound inside a massive metal sphere."*

I still assert that it is pure, wonderful, unadulterated, magic.

But, I'll let clearer heads prevail. From her speaker bio on Ted: "Composer JoAnn Kuchera-Morin works on the Allosphere, one of the largest scientific and artistic instruments in the world. Based at UCSB, the Allosphere and its 3D immersive theater maps complex data in time and space. Kuchera-Morin founded the Center for Research in Electronic Art Technology (CREATE) and has been the director since its birth in 1986. In 2000 she began work on a Digital Media Center within the California NanoSystems Institute at Santa Barbara. Her fascinations include gestural interfaces for performance and the expression of complex data in nontraditional forms."

Check it out: http://www.ted.com/index.php/talks/joann_kuchera_morin_tours_the_allosphere.html

If you need incentive to follow that link, consider this: researchers stand on a bridge in the center of a large spherical room/device and watch data. Imagine flying through the brain of one of the researchers and hearing blood density levels sung by computer agents. Higher density means higher pitch and more brain activity.

* From "About this Talk" on Ted found at the check it out link above.

Friday, April 10, 2009

Friday's Wonderful Find

I was following links yesterday and found sites that are doing exciting things with Internet data and data sharing. They are authored by Jonathon Harris and Sep Kamvar.

On Feelings

Harris and Kamvar have created a tool that mines the most common blog sites for "emotional" words. Every 10 minutes, their tool looks for entries that contain emotional keywords in a set of 5000 different words related to emotion (e.g., angry, love, lost, stupid, perpetual, futile, etc.) The full list can be found here.

Then, once it has found and plucked out the emotion-laden sentence, it mines the author's blog profile to get public demographic data about the blogger: name (pseudonym), gender, date and time of the post, location, age, etc. and mashed that data with available data such the weather at the blogger's location to create a database full of information surrounding that blog entry.

They have created a number of different pieces that use this same data -- exactly as I want to be able to do for mine. Some are:

We Feel Fine

Lovelines

Even better: they have created an API (Application Programming Interface) that allows the public to use the data they have collected which they've made available under the Creative Commons Attribution-NonComercials-Sharealike license.

Twistori is one award-winning site inspired by WeFeelFine. More about them and their ilk another day...

On The Use of Language

While wandering their site I found a source for one of their pieces (Word Count)-- the British National Corpus. From their site, the BNC is described as "a 100 million word collection of samples of written and spoken language from a wide range of sources, designed to represent a wide cross-section of current British English, both spoken and written."

There are a number of options for using this data. Harris and Kamvar used it to do frequency analysis of words in proximity to one another.

Buried somewhere in my art journal and in an old assignment is a piece that reads in listserve emails from listserves that I subscribe to. Their natures are completely different -- a technical listserve and (if memory serves) a bicycling mail list. The project was to read in the words used and compare their frequency of use between the two sources, representing them using side-by-side network diagrams or some other mechanism to highlight the differences and similarities in the use of language between these two populations of emailers.

I executed a proof-of-concept piece as an piece in Art 410 or 412 with one listserve.

It's exciting to find a resource where I can consider their use of language against the larger English-speaking population (UK English? American English?). I may have to resurrect that piece. I've been keeping those listserve emails so I have years worth of data to use.

The Dreaded RSI and My Adventures in Kaiserland

I've been scanning and scanning and coding and coding and it has done me in. I code for my job, code for my Software Development class and code for my Art class.

No kidding: I've started coding in my dreams!

So my body has finally rebelled: some time last week my right wrist started burning - constantly. And then I noticed that when I flexed my hand backwards, a white bump-y protrusion popped out just on the inside of my wrist below the heel of my palm. Actually, not one but two. The right's larger than the left.

I suspect it's more skin trauma from rubbing, rubbing, rubbing, resting and rubbing more on laptop edges and keyboard wrist pads. But I couldn't take a chance. The constant burning is what's the "red flag" part.

Long story short, I finally called Kaiser and for $15 they saw me within two hours, gave me some advice, the Dr. consulted a hand surgeon and they sent me to X-Ray. This is a long way from the Kaiser that made me wait 6 weeks for an appointment to examine an ankle sprain years ago

"Sorry, Ms. Mooney, we can't determine what's wrong. There's no swelling or pain."

"No kidding, Doc, it was six weeks ago."

So yesterday the Doc at a much improved Kaiser said, ice 3x a day, Motrin 600 3x a day, rest my wrists for six weeks, bandage it to protect from rubbing and we'll see. Could just be the start of a cyst and not an RSI. We'll follow-up then if it's still bothering you.

"Oh Doc, that's not an option. I have a month left of school and I have to do a lot of programming in the meantime."

So, following his advice for mitigation in the meantime, I'll be wearing a wrist brace and waiting it out. I've already been bandaging it and looking like I'm on a weekend furlough from suicide watch, self-consciously pulling my sleeve down over my wrist again and again to hide the "evidence."

Darn, I was hoping it would be something they could aspirate or medicate and this would be over!

Monday, March 30, 2009

Context Free

So I had a good re-discovery this weekend: while rallying troops and supplies to go to IC's (GC's youngest) lacrosse game, I grabbed one of my latest issues of Make magazine off the back of the toilet. It apparently has become GC's favorite reading material during his evening "constitutionals." At least now I always know where to find it.

Anyway, back at the farm, as I was laying on my blanket in the sun leafing through the magazine during time outs, I found an article on Context Free. It is an IDE/Programming Language sort of thing for creating abstract generative art things that are just plain nifty.

I had completely forgotten about it. It is simple to get started using it so some years ago when I found it used to create desktop images for my computer with it. I suspected (and still do) that it was capable of some more sophisticated stuff but, alas, I never got that far.

I downloaded the latest version last night after I finished reading the article (Make 17). I'm hoping to use it to create one of my Variations for my Senior Show piece.

Variations on a Theme

So what have I been doing about Variations? Well, I think a lot, panic a little and then think some more. When the panic becomes overwhelming I scan more receipts. It's mind-numbing but it calms me. LOL

The happy thing is that this weekend when I needed mindless work to do, I went up into the garage and I found the receipt files I had organized before I moved North. In it were much of my 2006 receipts. I have enough that I think I can do Variations for December 2006.

If it doesn't look good, I'll do October - December 2006 to get enough data.

But there's the question of it being bi-coastal data. I don't know if that'll be interesting or distracting. Only running a few pieces through will tell.

Pushing Myself into the Uncomfortable Place

Feeling a bit panicked about the Senior Show coming up. I've never done print work before so I've taken a leap of faith -- more accurately painted myself into a corner by committing myself to doing something that dumps me right outside of my comfort zone.

The "easy" part: I've decided to create a "finished" version of my pixel-y, generative piece using my receipts. I had planned to have users be able to interact with the piece to reveal the data behind the pixels that are displayed on the screen. I have a ton of different pieces that I have in my notes to create using this dataset. However, I worry about trying to throw everything but the kitchen sink into a piece (and losing any chance at having a strong concept behind it). What Rick and I used to call doing "parlor tricks" instead of solid conceptual work.

But I really want the piece to be more than a simple abstraction. Some classmates gave me some really great ideas about ways to incorporate my receipt images and data into the interface. Hopefully I can get some of that working in time for the show.

The scary part: That's the second piece I submitted. It's called "Variations on a Theme" mostly because I was fried and not feeling creative. At this point, the concept is in the relationships of the source data, not in the execution of the pieces. So anything I might decide to call it might not make sense.

The piece itself is a set of digital prints that all use the same base data but represent it in different ways. I like the idea of having four (three?) completely different pieces that seem unrelated perhaps at worst or have some dynamic tension at best (likely for a later iteration, however) that all are based on the same data.

Part of the problem is that I have to learn how to do graphics representation of data in at least three different platforms with three different coding tools (API's). That's a lot of learning. Then I need to apply an aesthetic or a concept on top of that. That's a tall order.

In the meantime, I have to decide on deliverables that I can complete on time. I'm focusing on repurposing of data for now and I'll strengthen the conceptual part once I've gotten that done.

My pixel-y project (called "Open Secrets" -- thanks Paula!) may only be an abstract generative piece with no user/viewer interaction for the show. Once I get it working and the abstraction pleases me, then I'll set it aside.

The Variations project (so far) will be some few of these 1) a literal mapping of purchases during a fixed time period on Google Earth or Google Maps, 2) an organically shaped abstraction using Context Free, 3) a static representation of that time period using the model for Open Secrets, and 4) piece(s) that shows connections between purchases either spatially, temporally or categorically.

We'll see what I come up with as I play in those four sandboxes because ultimately, I want four distinct pieces that look very different but use the exact same data. And I want them to be purdy.

Tuesday, March 17, 2009

Work in Progress

Cycling version with date label (large boxes): http:\\www.geezerwicket.com\cia\v3\

Raw text files: http:\\www.geezerwicket.com\cia\v1\receipts.csv

Pattern version: http:\\www.geezerwicket.com\cia\v2\

Thursday, March 12, 2009

I am interested initially in two kinds of mapping of data this semester. The first is a physical, geographic mapping of purchases. By capturing purchase locations (and perhaps later production, manufacturing or agricultural sources), I can root the purchases in space and create spatial relations among them that bear no relationship to their inherent similarities or differences. I wonder, as I undertake this effort, whether I won't see patterns emerge among seemingly unrelated datasets. Do I buy more food when I'm in the city or the suburbs? Does more entertainment spending happen on the West Coast or the East Coast.

Similarly, by capturing dates (and times when possible) I can relate the purchases temporally to things like political events, personal life changes, seasons, etc. What interesting relationships pop up with that kind of review? This, however, will wait until later.

The second type of mapping I want to explore is conceptual mapping of data. How do my purchases group or relate to one another? Is spending greater on staples or on entertainment? If I take it further and characterize those purchases further to describe how I felt when I made the purchase or who the purchase was for I can examine data relationships differently. Do I buy things in brick and mortar stores or at farmer's markets and thrift stores? Are purchase obligatory or discretionary? Which ones make me happy and which ones do I resent?

This approach to examining my spending lends itself more towards conceptual mapping schema like network mapping or to serve as fodder for generative art pieces. They could introduce random data to influence audio pieces or alter color palettes in video, for example. As a result, these pieces may or may not reveal the underlying topic of interest (consumerism, purchases, etc.) but may still expose patterns in the underlying data.

Getting Underway

So the first order of business, after having begun the onerous process of scanning receipts, verifying OCR and filling in missing data from the receipts themselves is identifying how I want to look at this data. Specifically, what other ways do I want to review the data other than just geographically or by totals? What questions do I want to ask now? Later?

I've decided that one way to explore the possible questions that I might ask is by beginning the work of categorizing or classifying the receipts. Already, as I've begun to review the data pulled via OCR from the receipts themselves, more ideas have come to me. I expect that to continue as I dive deeper into this process.

I have a couple of pieces that suggest looking at data as it relates to other data. My Monument to Applied Mathematics is a dynamic three dimensional sculptural graph illustrating the results of how related groups of statistical data influence one another. How does public spending on education influence tax revenues or juvenile crime, for example.

For this semester, I am examining my own spending behavior over several years. Rather than analyze the data to learn something from it, I am interested in numerical relationships between spending data and what kind of art pieces they produce when some aesthetic behavior or characteristics are applied to those relationships.

In general, as a database professional I am interested in data, data relationships, getting meaning from data (information or knowledge as opposed to more data), as a mathematician, patterns in those data appeal to me and as an artist I want to see how those data and patterns can create experiences or art pieces that reveal something about the data or simply just create something “pretty.”

Before any of these relationships can be examined or exploited, the data must be categorized in order to reveal relationships.

So, the next step for me, to serve my longer-term interests is to understand the ideas behind the classification of data.

Ontologies, Categories and Grouping, oh my!

According to Wikipedia, in general ontology means “ the study of the nature of being, existence or reality in general, as well as of the basic categories of being and their relations.” In particular, in the context of my profession, information technology, they define it as “a formal representation of a set of concepts within a domain and the relationships between those concepts. It is used to reason about the properties of that domain, and may be used to define the domain. “

I like the layered meaning because the latter, the formal representation, lends itself to my immediate interests. Once I can exploit the tools and techniques for creating an ontological view of my data, I can abstract it further in a way that is more in keeping with the philosophical definition. The code, tools, mechanisms, etc. will allow me to create representations of the information buried in and related to my receipts in a way that they allow me to explore the study of myself or my existence in this culture in a way. Issues of consumerism, privacy, etc. all seem rich fodder for examination once the data is collected and classified.

With the rapid advancements in technology and a greater understanding of the power of well-designed information systems, more and more organizations are looking hard at how they manage data, particularly how they get information from the data that they have. With this growing awareness and commitment to managing data as assets rather than as artifacts of transactions, more work is being done in the area of examining ontologies as they relate to business data and practices. They're being used extensively in biomedical research to facilitate data sharing and cross-domain research, for example.

In my work I am spending a good amount of time reviewing the projects and tools that have been developed as a result of an effort of the National Cancer Institute (“NCI”) of the National Institutes of Health (“NIH”) to allow data sharing far beyond the bounds where that has historically occurred. Fears about intellectual property, incompatibilities among proprietary systems and data formats as well as immature data communications technologies have been a “perfect storm” of hurdles that create disincentives to explore data sharing. However, growing citizen awareness of data management possibilities as well as political pressure to manage data more effectively has created a climate where government agencies now require that data collected for research funded in whole or in part by the US Federal Government must be available to be shared with other researchers.

To share data, however, there must either be standards for data collection itself or for communicating characteristics of data (“metadata”) so that potential users can translate it into a format where they can integrate it into their own systems or data sets. As these vocabularies or data dictionaries are developed and adopted on a large scale, referencing those definitions becomes an information management problem in and of itself. How do I find out if anyone has information on genetic markers for Hispanic children below the age of 15 or occupational data for their parents in a pool of tens of thousands of data elements unless those metadata are categorized in a way that facilitates finding that data. Text searching, after all, is still an ineffective way to mine a large pool of data to get a list of search results that having real meaning.

Effective categorization of data allows users of that data to navigate large data sets efficiently in order to find the information that they need. This is where ontologies step in.

Glossary: http://www.jfsowa.com/ontology/gloss.htm

Standards and Standards Bodies

NATO:NCIS T2 Committee on Information Interchange and Interpretation
DAML: Darpa Agent Markup Language http://www.daml.org/
OWL: Web Ontology Language http://www.w3.org/TR/owl-ref/

(is-a, has-a)

History

Aristotle's Categories

substance,
quantity,
quality,
relation,
place,
time,
position,
state,
action,
being acted on

Symbolic Logic and Logic Tools

Ramon Lull (1235 – 1316) from Majorca – Lull (“Lullus”) posited that each area of scientific discovery had a root number of fundamental principals and terms that can “be accepted without further inquiry.”¹

"The diagram ... is the tree of nature and logic by the thirteenth century poet, philosopher, and missionary Ramon Lull. The main trunk supports a version of the tree of Porphyry, which illustrates Aristotle's categories. The ten leaves on the right represent ten types of questions, and the ten leaves on the left are keyed to a system of rotating disks for generating answers. Such diagrams and disks comprise Lull's Ars Magna (Great Art), which was the first attempt to develop mechanical aids to reasoning. It served as an inspiration to the pioneer in symbolic logic, Gottfried Wilhelm Liebniz.²

Modern Ontologies

A new development in data management that can be useful for navigating large datasets capitalizes on the development of these ontological mapping schema and other mechanisms to allow computers systems to do part of this work for us. It is the emergence of the Semantic Web.

Protege
OWL
Content Navigation
Information Architecture
Knowledge Interchange Format (KIF) http://www.cs.umbc.edu/kif/

The Semantic Web

The inventor of the Internet, Tim Berners-Lee proposed “a web of data that can be processed directly and indirectly by machines.”³ Specifically, Lee says “[t]he Semantic Web is not a separate Web but an extension of the current one, in which information is given well-defined meaning, better enabling computers and people to work in cooperation.”⁴ Currently, the web is a rich source of material that is largely made available through the use of browsers for people to read, interpret, amend, etc. This last part of the scenario is the critical one – the traditional corpus of web content requires that it be read and interpreted by people. The Semantic Web, on the other hand, proposes an evolution of the web as a platform where data available for computers to analyze, assemble and use to create meaning that can then be used more effectively.

Responding to Berner's Lee's proposal, the W3C, the governing body of the World Wide Web, created the Semantic Web Coordination Group and a host of other working groups to conceptualize how this next generation Internet should work and to define standards that will allow diverse communities to work together to realize this vision.

The Semantic Web Activity site (http://www.w3.org/2001/sw/ ) describes what this means in a way that I find easy to understand. They say:

The Semantic Web is a web of data. There is lots of data we all use every day, and it is not part of the web. I can see my bank statements on the web, and my photographs, and I can see my appointments in a calendar. But can I see my photos in a calendar to see what I was doing when I took them? Can I see bank statement lines in a calendar?

Why not? Because we don't have a web of data. Because data is controlled by applications, and each application keeps it to itself.

The Semantic Web is about two things. It is about common formats for integration and combination of data drawn from diverse sources, where on the original Web mainly concentrated on the interchange of documents. It is also about language for recording how the data relates to real world objects. That allows a person, or a machine, to start off in one database, and then move through an unending set of databases which are connected not by wires but by being about the same thing.

Scientific American creates a compelling scenario ⁵ that illustrates what the development of a Semantic Web would look like in the future as people conduct the regular business of their lives.

Bibliography

Free Will?

Ironically (or perhaps it was just as I suspected), as I consider the questions that I can explore with my artwork, more pop up all the time. I've been immersed in ideas related to what you can know about someone when you know how they spend their money -- particularly in a culture as commercial as ours.

I hope to see patterns and connections emerge from the data as I play with them but another delightful connection popped up as I thought more about where these projects may lead: What happens to self-determination when you are laid bare in a world full of experts who want to influence your opinions and behaviors? It has been a serious concern of mine for quite a while and I realized this afternoon that it is a natural question to ask as a result of the work that I am doing. There's a connection I didn't anticipate!

So here's my question:

Given the power of persuasion and the sophistication of market researchers, jury selection experts, political strategist, and other experts who understand how to influence opinions, preferences and behaviors without the knowledge and consent of the Manipulated, what does more data mean about people's self-determination? How do you combat targeted messages crafted by anonymous experts who know your motivations, prejudices, needs and habits – particularly when you don't know that you're being exposed to those messages?

I have serious concern about the influence that jury selection consultants have on the execution of Justice in this country. Similarly, I am wary of what we get as a nation when highly educated, highly self-serving politicos exploit the much-too-large population in this country of citizens who are not very sophisticated about media and message, who don't have well developed critical thinking skills due to poor or little education or those two have cultural or other prejudices so well-developed that they can be exploited when messages are crafted that tie dubious conclusions to strongly held belief systems. The more information those experts have at their fingertips and the more the quality of our education erodes, the less free-will and self-determination we really have as a country. And inherent in that dynamic is the fact that the Manipulated will be entirely unaware of how they have been compromised.

Tuesday, March 10, 2009

Some Links

Preliminary Receipt Export: http://spreadsheets.google.com/ccc?key=prZxcYlb93Ef797YJEva-Bw&hl=en

Research Paper Notes: http://docs.google.com/fileview?id=F.70b69cda-96f1-480b-a26e-d1c727177912&hl=en

Neat Receipts: http://docs.google.com/Doc?docid=dcvj5mpc_33ffvc25gp&hl=en

Network Diagrams

Parc

Friday, March 6, 2009

Throwing Wrenches into the Works

As I write I sit in the jury "lounge" of the Marin County Courts. It's a gorgeous building, designed by Frank Lloyd Wright, one that I've wanted to be able to explore for years. I'm a huge fan of his architecture.

Unfortunately, I didn't know that the first real opportunity to check it out would be as an "inmate." Well, strictly speaking I'm not an inmate. I'm a prospective juror for a 4-5 week felony criminal case. And they're calling everyone it turns out. Budget's too tight to call tons of jurors and let loads go free for another year. So the bulk of the pool were released for the day roughly an hour ago and those of us who are claiming hardship have to wait to see the judge for a decision about whether we can be excused or not.

So I will have given an entire day to the Marin County Courts regardless. That's fine. I expect to do that periodically. I just don't want to lose the day AND have to lose 4-5 weeks on top of it.

After we watched an "inspiring" film that was "designed to get us excited about doing jury duty," one with music, the Head Jury Coordinator opined, that is "heroic yet tugs at your heartstrings," we were read a list of legally defined standards for "extreme hardship." Full-time studies is one, part-time is not. Pre-paid travel is one, pre-paid tuition is not.

I'm pleading my case and hoping for the best. At worst, I think I'll be excused during jury selection. Don't think defense counsel will be eager to have a pissed off citizen who resents the fact that some punk who wawnts what they want when they want it can rob me too of $1,300 of hard-earned cash and 4-5 weeks of my time -- not to mention the 5 weeks of hard work I've already put in for this term that will be wasted. I'm almost positive that will get me excused.

But we'll see.

In the meantime, 1300 receipts scanned!

Monday, March 2, 2009

Serious Progress

611 receipts down! (and thousands to go)

I spent a lot of time this weekend scanning in receipts, proofreading the OCR results for dates, totals, vendors, etc. The NeatReceipts scanner does a really good job. It does seem to have trouble distinguishing between 2008 and 2003, giving me the latter far too frequently for the former, but it generally does a good job.

In fact, it does a great job. Some of the things I want to explore with my work (eventually) are privacy relted issues. What can VISA or MasterCard deduce about me from reviewing my purchases. It turns out that the NeatWorks software that comes with the scanner -- the part that actually does most of the heavy lifting -- recognizes the words Debit, Visa, Cash, etc. In addition, it also recognizes the pattern that we've all become familiar with recently ****-****-****-2222 that shows the last four digits of our credit card used. The NeatWorks software identifies not only the method of payment more often than not, but can also identify which card I've used by looking for that pattern.

It's pretty awesome.

My one complaint so far: I haven't seen yet how I can customize the data entry screens so that I can extract more information from the receipts as I'm proofing them (desc or reason for purchase, vendor address and phone, clerk name). It looks like I'll have to run through them all again manually and capture that information either in a customized editing screen in NeatWorks (if that's possible) or in my mySQL database.

Friday, February 27, 2009

Oh the magic of technology!

So yesterday, having just gotten my paycheck, I headed for Costco to stock up on toilet paper and storage bins. As is my wont, I wandered through each of the aisles with my cart, weighing possibilities, and fantasizing about how each new wonder I found would alter my life for the better.

It's a game I play, put something in the basket, think about how wonderful or helpful it would be and then an aisle or so later, after having considered all those options and then how better or more responsibly to use my money, I return to the scene of the crime and return the merchandise to the shelf from whence it came.

Yesterday was different. Well, not entirely different but different all the same.

I toured the TV aisles near the front of the store ("oh that 46" Sony seems wonderful"), passed by the monitors and securities system ("maybe that will help us find out who is entering our apartment when we're gone"), then turned down the office supplies ("mechanical pencils for Ian?"). Each time I pondered and then continued on.

That is until I turned the corner as I left the office supplies aisle. Turn left -- mattress, turn immediately left again and there it was...

My NeatReceipts scanner. Woo Hoo!!

I've mentioned it in previous posts but, in brief, this is the portable bar scanner that is one of a suite of hardware and software from NeatCo which allows me to scan images of receipts and to capture data from them automatically.

It uses OCR technology and other logic to pick up method of payment, credit card last 4 digits (xxxx-1234), total sales, vendor names, dates of purchases etc. and it will both connect to Quicken for data exchange with that software but will also allow me to export data from the receipts into Excel or a .csv file that I can then pull into my project database.

Last night I scanned 88 receipts and captured dates, totals, tax and vendors for all of them.

88!! That's just crazy talk.

Tonight I try to customize the database to capture my other data like time, employee, databaseIDNo, etc.

I wonder if I can code a system to automatically update an online database with purchase data as I make the purchases (the scanner is very portable and comes with its own case) so those data reflect immediately on a web interface.

Thursday, February 19, 2009

Class Presentation

I've been primarily doing research and groundwork on processes and technologies to do my work this semester. (Terms shown in parenthesis are labels attached to related posts in this blog.)

Processes (process, artpractice)
1. art practice
2. project execution
Scanning receipts
1. manual scanning (process)
2. neat receipts (technology)
Receipt Data Store (technology)
Presentation Technologies (technology, mapping)
1. Google Maps - Art610 Magee
2. Google Earth - sample trip
3. SketchUp
Mapping (technology, mapping)
1. Theory
2. Technologies

Some Mapping Resources

Mapping resources that have some kind of GIS or temporal data component:

TimeMap "TimeMap is a mature, Java temporal web mapping/animation application. Use as a standalone web site or CD-ROM app, or as a sophisticated client-server system using an XML or SQL metadata directory to source distributed, incrementally-loaded spatial data."
iDataMap "iDataMap software allows users to map economic and demographic data with different geographical boundaries on the same map. Our software contains an algorithm that allows users to create maps that represent one or more economic and/or demographic variables from a particular data source. iDataMap software also allows users to overlay two maps so that the user can visualize how variables from two different data sources interact. "
Starmapper: might be useful for non-literal interpretation of map data. "Starmapper is a maping utility that can read a map of "nodes", some files with "reports" and draw some pretty pictures. Originally for Stars! game, now it's Stars! and beyond! (you can make your own data model and plug it in)"
Geodoggy: "GeoDoggy (Perl CGI) is a web-based interface and national map dataset that enables users to render maps of specific portions of the US, DC, and US territories. The data is derived from the US Bureau of the Census Public Domain Tiger Line (TM) dataset."

Things about Google Earth

Things I've learned so far:

I can import waypoint, tracks and route data from a GPS system.
New placemarks are created using lat and long data.
The data points imported into Google Earth from Google Maps have lat and log properties extracted and available in the waypoint's "Properties" dialog box.
I can create folders which could allow me to organize data points by year of purchase.
Google Earth animates zooming in to Street Level image display.
GE Pro allows GPS data import into the map.
GE Pro allows real-time GPS tracking
GE Pro allows importing of spreadsheet data with up to 2500 points.
GE Pro allows GIS data importing (ArcGIS?)
Google has a product called "SketchUp" which allows us to create extruded 2D and 3D graphics. This could be useful for representing multiple events (purchases) at a single location if it can be mashed up with Google Maps or Google Earth.

Things to find out:

Can I automate "zooming" in to data points when Street Level images exist for them?
Can I export placemarks I create in Google Earth to my My Maps map in Google Maps?
Is it possible to programmatically alter the speed of movement from one waypoint to another based upon some piece of data? This would allow me to simulate time differences among purchases.
Find out how "real-time" GPS tracking works.
What kind of GIS data are out there in the public domain that I may use in conjunction with my receipts?
Get feature set and free vs. paid license feature comparison for SketchUp.

Resources:

Google Earth Community - http://bbs.keyhole.com/ubb/ubbthreads.php/Cat/0
Tutorial for creating custom kml's -- http://earth.google.com/intl/en/outreach/tutorials.html
Instructions for putting a SketchUp graphic in Google Earth -- http://sketchup.google.com/support/bin/answer.py?hl=en&answer=36241
Temporal data mapping:
1. http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=1839378
2. http://www.esri.com/news/arcuser/1104/temporal.html

Google Maps data

I've been manually posting vendor locations into Google Maps using address info from receipts. On the screen, just above the map graphic, is a tantalizing link labelled "View in Google Earth." But instead of seeing some spiffy Google Earth magic, when I clicked it, all my browser did was download a file named Art610 Magee.kml.

So what happened?

Well, I have Google Earth installed on this computer but it didn't automagically launch. I'll have to fix that. In the meantime, I'm installing the Google Earth browser plug-in to see what happens when that's in place.

The cool part: Give me a file and I need to know what's in it. So, I opened the .kml file in Notepad and learned that it's just an XML data file that uses a data format that Google Earth recognizes.

Here's the DTD that I can discern from the file:


<kml xmlns="http://earth.google.com/kml/2.2">
<Document>
  <name></name>    /* the map name */
  <description><![CDATA[]]></description>
  <NetworkLink>
    <name></name>  /* the map name */
    <Link>                  
      <href></href>  /* URL to location on map */
    </Link>
  </NetworkLink>
</Document>
</kml>

That means a couple of things:

I can generate my own .kml files automatically from data so that they can be opened by Google Earth.

I can continue to map my vendors in Google Maps and upload all the URLs and other data shown below into my database using batch processing. This simplifies data collection dramatically.

Unfortunately, it appears only to export one data point at a time. I have to see if it's possible to download the entire set of map points all at once.

Saturday, May 23, 2009

Monday, May 18, 2009

Thursday, April 16, 2009

Wednesday, April 15, 2009

Friday, April 10, 2009

Monday, March 30, 2009

Tuesday, March 17, 2009

Thursday, March 12, 2009

Getting Underway

Ontologies, Categories and Grouping, oh my!

History

Modern Ontologies

The Semantic Web

Tuesday, March 10, 2009

Friday, March 6, 2009

Monday, March 2, 2009

Friday, February 27, 2009

Thursday, February 19, 2009

Conceptual and Information Arts Searching

About Me

Tags

conceptual arts-y things

technology things

beautiful things

Blog Archive