Linked from Makezine

We were linked from the Makezine site. We had a rush of traffic as people came to expirement with the cross stitch generator. Hopefully this will encourage more people to use it and provide more cross stitch charts to others.

google base

I added a feed to Google's base index. I opted to use the tab delimited format bulk upload through ftp. It was similar to other feeds and required only reworking the froogle feed. I also reused the same ftp upload script that I used for froogle. Base only accepted the first 3000 products. I'm not sure what Google is at here since our products are not classified ads, nor local to a region. It looks like eBay is automatically being sucked in. We come up prominently for our primary keywords, and we have made at least one sale, so I am not complaining. I'm not seeing any benefit to our listings on either google or froogle yet. Froogle has turned into a steady earner for us and is certainly superior to both bizrate and dealtime as well as yahoo's product feed. I'm really looking forward to Froogle getting it's act together and adding merchant and customer reviews like eBay.

I started playing with Firefox 1.5's canvas element. I've been working on an image editor that I want to utilize for the chart generator. It's really slow on my old P3 733 I have at home, but at work it runs just fine on a P4. After that, there are a bunch of other features I want to add that I have not had time for.

Irises - Van Gogh

I just created a counted cross stitch chart of Van Gogh's Irises and put a copy in my hidden stash. Creating cross stitch patterns is tricky business particularly if you're lazy like me. I don't want to edit the image, perferring to just upload it and let the software iron out the wrinkles. There are several guidelines I use to pick candidates for charting. First is to avoid faces if possible. Sometimes they come out alright. But since the human brain has a special portion of itself set aside just for facial processing, faces don't tend to chart well. The reduction in the number of colors, even with an extended set of blended threads tends to reduce the quality beyond which we will accept. Second is to pick an image with a topic relevant to the target market. Lighthouses, wedding, nature, cats, etc. Impressionist art is my personal favorite. Then pick an appropriate background color if possible. White is the best background color. I'm not afraid of being minimalist and not throwing stitches over every weave on the canvas. Then drive the complexity of the image by determining the stitch count and the use of blended threads. A high thread count makes for alot of threading changes. A limited thread count makes for less detail and realism. A cartoon type of image is ideal for low thread counts. The same with black and white or grayscale images. If you decide to exapnd your pallette with blended threads, the complexity of the chart increases as well. I like to look for images on Google. There are other image search sites as well. The isolated stitch count is important too. If you want to make the chart easier to stitch, raise the count to something higher like 3 to 5. This will make the chart easier to stitch at the cost of reducing the detail of the image. You can achive maximum detail by setting the isolated stitch count to zero. I recommend zero if you edit your image before uploading it to preserve the human touch. Even though I'm too lazy to manually edit the images I use, software cannot replace the benefit of human editing. More on that another time.

RSS vs. Atom

I was looking at implementing a syndication feed for our catalog the other day. A XML feed allows data to be transferred between entities without the formatting text included. Or you can some of it, if you really want. It's structured data that can be easily parsed by software. HTML does not have this characteristic, but XML does. This blog for example has a feed. This was brought about by my seeing an increase in websites that include the "alternate" tag. I like to be the first craft site on the block to add a new feature, so I looked into it. The big decision is the feed format. There are currently two strongly supported formats for syndication. The first is RSS, which stands for Really Simple Syndication, or something else depending on who you talk to. RSS has been around for some time and you can view its history by clicking on the aforementioned link. The other format is Atom which is being pushed by Google. Both formats have specifications by industry standard groups and both are supported by a wide variety of different software programs.
But which is better? I decided on Atom. My reasons reach beyond just what Google tells me to do and has more to do with why Google is backing it. I think that the reason Google favors Atom over RSS is that Atom is better. "RSS is too complicated" is the basis for that. This is an over generalized statement but I'm hoping to boil the argument down without delving into overly excruciating detail. I have seen the new RSS 2.0 format, and yes it is simpler, but not simple enough and allows for considerable abuse. RSS is based on RDF, which is a really good way to store data and is used in software like Mozilla which is one of the reasons it is a really good thing. The downfall of RSS I believe, is that it allows for extensibility via RDF. This is normally a good thing and a very powerful feature. But when trading data between websites, it is unecessary. Since both websites must agree on the XML format prior to transfer, there is no way for the receiving website to interpret the tags that have been added to the format by the sending website. This doesn't make RDF or RSS bad, rather it means that they are not ideal for the use for which I'm intending. Atom lacks the overhead of support for tag extension and as a result is simpler.
On the internet, there is a trend that is usually true. In a conflict of two standards, one will win. Not all the time, but most of the time. By win I mean widespread adoption at the expense of the other. I think that RSS will be supported solely by some websites for a long time but will be the increasing minority. Many websites will support both for a while until they drop RSS and only support Atom to reduce maintenance. Some sites, probably the majority, will support just Atom. The pressure of market forces will apply increasingly over time towards Atom.
So I settled on Atom. I didn't do the actual implementation though. I don't store the last modified time for products and categories in my database which is a required tag for both formats. Including the last modified time was something I thought about during design, but discarded it as I couldn't see the use of it later on. My mistake. As soon as I add the extra columns to my database, I'll be able to finish the feed. Hopefully in time, product feeds from different sites will allow consumers to accurately comparison shop without significant overhead. I'm tired of the tab delimited format standard. Which makes me wonder, why doesn't Froogle take Atom?

Raster to Vector Conversion

There was a story on slashdot.org about different vector image editors. The story seemed to be brought up do to the ensuing hype about Firefox supporting SVG in a future release. Vector graphics support in web browsers is exciting to me since I have been working on raster to vector conversion software in Java for counted cross stitch patterns for so long. There appears to be a general consensus that a raster image can't be converted to a vector image. There isn't even an entry on wikipedia to any information regarding conversion. I've begun to think that there is a possible market for a service or product to do raster to vector conversion with the advent of browser support for SVG. There is an immense pool of raster images on the internet, but very limited support for converting them to vector images.
A search on Google displays a large selection of articles regarding region growing and image segmentation. The feeling I get is that the raster to vector conversion has been thought of as too processor intensive or RAM heavy to use brute force computation to solve. This might be no longer true with modern hardware. There is also an emphasis on real time processing of images which is highly time bound that doesn't fit the same market as the conversion tool I'm thinking of. Although admittedly, photo-realistic images in a vector format is not an ideal fit. But if you get past the idea that photos will get watered down by the conversion, or the file becomes too large, it just feels right to store all images as vector based. I can't back that feeling up with any evidence or hard numbers unfortunately.
Converting to vector images begs a couple of questions though. Number one is, can meaning be drawn for the vectorized image and stored for searching. I haven't solved this issue in my mind yet. It's easy to pull out lower end information from an image like colors and simple shapes. But pulling out "horse" from a picture of a horse is something else entirely. I know it can be done because my brain is doing it right now, but reproducing it in software is a something else. The second issue is video. This is the simpler of the two problems to solve. Video would be very processor intensive to encode it. The decoding process would be trivial though and could lead to smaller file sizes than MPEG possibly. I took a look at the format for MPEG and was stunned by how convoluted it is. I'm thinking that the industry is willing to take a hit on the resources and time it takes to encode a video in return for trivial decoding and a slightly smaller file size. Smaller, since only the difference between each frame would be stored to render the next one without dropping a refresher frame in every fourth time along with all the "super block" hoop jumping. If you look at what Macromedia has done with Flash, this approach just seems right too.
My primary inhibitor to working on this is hubris. I'm not Einstein in a patent office. I sell crafts on the internet. Who do I think I am to pull off something that others have dismissed as implausible? I've been looking around for someone who is doing this, and doing it well and I haven't seen it yet. If you have, let me know.

Ebay Copyright Violators

I think that there are flaws with the current copyright law. I do believe that those who create, deserve to get credit and compensation for their work. A common problem within the counted cross stitch industry is the proliferation of people who use the artwork of others for profit without proper accredidation. Fair use allows for non-commercial personal use but not for resale. Ebay, IOffer, and other auction sites have this problem. So let me share my most recent experience.

I found this seller on Ebay, selling charts based on artwork by Carrie Hall. What struck me immediately, was that the titles of the pieces were changed and that there was no artist recognition. I smelled something rotten. I sent an email to Carrie asking if this person had the right to sell the charts. While waiting for a response from Carrie, I decided to contact the seller as well. The following is the exchanges of email verbatim, including my poor typing. (warning: unprofessional language follows)

Dark Lilac: who is the artist for sea witch, winter watch and Sian?
Seller: cross-stitch-charts: why?
Dark Lilac: No problem. I found them. They will contacting you shortly.
cross-stitch-charts:good for you
cross-stitch-charts:whats your problem.
cross-stitch-charts: DID YOU JOIN JUST TO BE AN ASS?
OR HAVE YOU AL;WAYS BEEN THIS WAY?

I think the defensive attiture of the seller is indicative of guilt and ignorance. I feel particularly sorry for the buyers of this material who have been duped. Normally this would have set me off. But I was in a good mood after chatting with other stitchers on the HAED BB who cheered me up. I sent Ebay an email complaint about the seller.

Carrie did get back to me and said that this person did not have the right to sell her stuff and she would be contacting them. I'm waiting to see what Ebay does. At this time, the autions are still live with bids.

Yahoo Feed

The error message that caused the product images not to show up on yahoo's product feed was "Include a valid image for these product(s) by populating the "image-url" field". After I fixed the retail price to display double zeros for round number prices (9.00 instead of 9), then the images were picked up. Apparently an invalid image is a euphemism for something completely different. A couple of the products are still not showing images, but I'm giving up here and calling it good. Is accepting XML instead of tab delimited plain text too much to ask?

Product Comparison Feeds

I've done several product feeds for darklilac.com and here is a breakdown of each:

Froogle:
Froogle from the fine folks at Google is free. Due to the low cost, all the products on the website are uploaded. It takes a tab delimited plain text file via ftp upload. I wrote a shell script that is called by a cron job that does the ftp upload for me. It would be easier to just have Google do a periodic download from the website. Froogle is a must for every ecommerce site. If Google can add the ability to track consumer feedback, Froogle would be a direct competitor to eBay. Ebay does two things well, search across sellers and feedback. If Google gets their act together, they could do some very good things for the ecommercesphere. I have not yet figured out a way to do keyword optimization on froogle yet. Which is probably a good thing.
http://www.darklilac.com/frooglefeed.jsp

Bizrate (shopzilla): Bizrate has been around for some time and charge on a per click basis. They take a tab delimited file but don't require an ftp upload. Since they charge, I only allow a subset of products to be displayed. Only allowing targeted products (about 50) will decrease the daily charge. Bizrate is a difficult partner. They advertise on adwords and overture. Sometimes they will have the top 2 (two!) spots bid on for themselves. They do this to match whatever keywords are in their system to drive revenue. Because they compete on keyword bidding, they drive up the costs against their own partners. To combat this, my account is limited to a low monthly charge. I don't have a lot of faith Bizrate. The reasons involve the extra competition on adwords and overture, inflexible bidding (too high!), difficult to manage categorization of products and the delay to bring up pages. I'm guessing that the Bizrate IT staff had to pay an early adopter penalty to setup their system, which is why it needs a splash screen.
http://www.darklilac.com/bizratefeed.jsp

Dealtime (shopping.com) Dealtime is weaker than bizrate. They also generate fewer sales but don't compete on adwords and overture as much. They also don't require an ftp upload but put a requirement on the file extension which has to be .txt. This required me to tweak my webserver settings to treat .txt files as .jsp's. Also, the feedback for errors aren't as good as froogle or bizrate. Their categories are confusing like bizrate but the site response time is better. They also have inflexible bidding that is too high.
http://www.darklilac.com/dealtimefeed.txt

NextTag: I tried NextTag and couldn't get the upload to work. I haven't checked lately, so maybe they have fixed it.

Pricegrabber: They were not interested in carrying Dark Lilac's products.

Yahoo: I'm working on the Yahoo feed right now. Yahoo was late to the market and haven't done much to improve on their competitors. They also require an ftp upload like froogle or you can use your web browser to do it manually. Their bidding is inflexible, the error messages aren't clear either. The reposing time is good though. The jury is still out on Yahoo, but it isn't looking good so far. I'm still trying to get the product feed structured right and I can't tell what is wrong. I have some practice at doing this, so I'm guessin' it is not all my fault. Also, they require the filename to be data.txt. I find this fixed name requirement to be annoying as my dynamic files are served as .jsp. I know I could do a url rewrite, but that is more work than I am interested at the moment.
http://www.darklilac.com/data.txt

Their is a big space in this market that someone will fill. It's a big hole and a very profitable one as well. The key elements seem to be:

• Allow cross vendor searching by keywords like froogle
• Allow RSS style feeds that are sucked right off the vendor website
• Allow a configurable url for RSS feed
  
• Allow presentation values in the feed for display on the comparison site
• Allow flexible bidding
• Allow fixed price or auction style purchases
• Consumer feedback system for all vendors like eBay has
• Keep the keyword trickery to a minimum like froogle (but don't be random)