Changes, Fixes, and Feature Additions

An ongoing (and non-exhaustive) list of changes and improvements to Spendabit.

October 2022

Implement the beginnings of a specialized page for coffee, which will hopefully prove as a useful pattern for additional product categories.

August 2022

Improve ranking of gift cards in cases where you search for a particular brand, like Amazon or Red Lobster.
Finish work on an internal system that will help us review merchant profiles, which should help us to keep said profiles in better shape.
Rework our captcha implementation (used to cut down on spam to our contact form).
Make more small improvements to our “live tests”.

July 2022

Fix a problem whereby counting products for some merchants (those with lots of out-of-stock products) would be really slow.
Make some improvements to product-identification engine, with particular attention to results for “coffee” (which still needs a lot more work).
Address a bug with our invoicing tool that could occasionally lead to problems with outdated invoices.
Optimize the way we calculate popular searches, to speed up a bit in cases where we don't already have it cached.

May 2022

Continue to add to our Crypto Services section.
Make some modest improvements to our in-house tools for managing merchants.
Fix a bug that could potentially lead to problems for merchants when topping up, in cases where they have extra, legacy subscription records on file.
Spearhead work on an experimental background process/daemon that will hopefully help us to have fresher product inventories for some merchants (e.g., Newegg).

April 2022

Make some improvements to internal merchant-management/setup tools.
Fix a bug with generating links for Newegg.
Add support for SVG-format favicons.
Improve our invoice tool for cases where payment is made by Lightning Network (via OpenNode).
Add page for wallets.

March 2022

Begin work on some internal tools to help review merchant profiles.
Fix a bug that would prevent indexing of products that had (for example) a tab in their name.

February 2022

Rework the way we count merchant products, avoiding slow queries.
Fix sorting of blog posts.
Add new admin tool for adding merchants more easily on the back-end.
Add some more testing around on-boarding.

July 2021

Implement front-end support for filtering by “currency” (finally!). I.e., now you can find products available for Litecoin, Ethereum, Bitcoin Cash, Monero, etc.
Fix “scrunched products” on review-products screen of Onboarding Wizard.
Fine-tune wording/styling on “install back-link” page of Onboarding Wizard.
Improve our scrape-locking mechanism, used to avoid multiple scrapes/crawls happening simultaneously for the same merchant, to detect scrapes that have been abandoned (canceled, crashed, etc).
Make bug-fix to our product-identification engine to avoid crashes on some large (but not unprecedented) BTC-based prices.
Fix bug whereby blank merchant profiles could show up above search results.
Fix issue with regard to the way we calculate when it's time to re-crawl/scrape merchant websites.
Fix bug that was affecting our cache-update mechanism, whereby an id column was overflowing.
Update some URLs on our Press page.
Make some modest progress on our blogging tool.
Add new admin tool to help us reset passwords.
Implement a new revision to our internal web-forms library.
List crypto-currencies accepted by merchant on their profile page.
Rework the way we do in-memory caching for some data-types (e.g., merchants).
Do some very preliminary work on product categorization (hopefully to become functionality for browsing products by category, and more).
Make some modest improvements to our “live” test-suite.

June 2021

Implement a major overhaul of the caching layer to our search-engine, which should lead to substantially less strain on our system to keep the cache up-to-date.
Add lots of rules (mostly hard-coded queries, for now) to fix obvious typos and weed out spurious junk, to further decrease the load on our caching back-end.
Perform additional housekeeping and bug-fixes on caching back-end.
Revamp the “Free Plan” branch of our Onboarding Wizard to both improve the UI/UX and lessen the requirement (just ask for any-old back-link).Add some additional validation logic for our Onboarding Wizard forms.
Continue to refine our product-detection engine, in particular with regard to determining product availability.
Continue to improve our product-identification engine, including some particular attention given to identification of TVs and TV-related accessories, computers vs. smartphones, t-shirts, and coffee.
Add additional validation to our Search Partners signup form.
Fix a bug with our reset-password form.
Disable tool to allow merchants to set their own “tags”, as it has been leading to more problems than benefits.
Make some modest improvements to our admin tools for commissioning new merchants/accounts.
Tweak sitemap-generation a tad, to avoid handing over some more junk queries to web-crawlers.
Do some work related to dynamically identifying the “industry” for merchants (with no concrete application for production yet, but may soon be useful).Extend our “live” test-suite a bit more.
Create Telegram channel and add link for it to our social icons.

September 2020

Rework the back-end logic to our Onboarding Wizard to put us on sturdier ground (e.g., to avoid incomplete merchant records in cases where people don't complete the process or get hung up somehow).
Fix way our search back-end handles the currency parameter, paving the way for our (long overdue) support for alt-crypto-coins (e.g., BCH, ETH).
Continue to work on the problem of some websites blocking us (Newegg, especially) when attempting to checkup on products, look for new products, etc.
Improve our resilience to falsely concluding a product is defunct in cases where a website is giving back an error response.
Add some bare-minimum validation to our “choose currencies” form.
Rework our HTTP-client stuff a bit, particularly with regard to request-throttling.
Make additional improvements to our schema-migration tool.
Make some modest improvements to our subscription-management and onboarding administration tools.
Continue to improve our “live” test-suite.
Perform some tweaks to subscription-related testing infrastructure, to better avoid false-negative test failures.
Make some minor tweaks to our product-identification engine.
Fix a bug in our plural-to-singular normalization code to avoid breaking already-singular words in some edge cases.
Tweak and improve our helper-scripts for standing up new app servers.

August 2020

Continue to tinker with parameters for “on-demand scraping”, aiming to better avoid remote blocking/blacklisting.
Add beginnings of support for pulling in advanced product attributes (usually labeled as “specifications”) when “cultivating” products... which helps us to better identify products, and will eventually be used to provide more functionality on the front-end (e.g., advanced product filtering).
Improve our ability to pick out the correct/preferred favicon, in cases where there are multiple <link/> tags referencing images.
Implement some measures to avoid spam/nonsense coming in via our Onboarding Wizard.
Refine some of our tools for helping new businesses get on-boarded.
Make some modest improvements to our testing infrastructure.
Perform some internal refactoring to further move us over from org.joda.time to java.time.
Continue to tweak our outage-detection layer (though it's still a bit of a hack-job and needs a good re-think at some point).
Make the usual improvements to product-detection and product-identification code.

May 2020

Add initial support for “on-demand scraping”, which should greatly improve the “freshness” of our results (only enabled for Newegg to start).
Fix bug in dead-product detection that was leading to lots of products (those for Overstock in particular) being improperly identified as “dead”, and thus deleted.
Make some tweaks to product image-detection to prefer higher-resolution images when available.
Perform (routine) improvements to our product-identification engine, improving support for dealing with inconsistent use of compound words in particular (e.g., barstool vs. bar stool).
Refactor the “preprocessing” logic/code for our product-identification engine.
Begin to consolidate the schema, and associated handling code, for our “Directory” (service-style) businesses and traditional, “e-commerce” businesses (ones with products).
Tweak our data-dump (e.g., CSV) importation logic to store a better version of product description (which will help product-identification).
Implement initial version of an administrative tool for managing on-board attempts (e.g., following-up with businesses that never finished the process).
Make some improvements to the “plumbing” that makes aforementioned tool possible.
Remove requirement to provide an address for service-style businesses (non-eCommerce) when on-boarding (for now, at least).
Continue to build out our database-migration system.
Add hook to invalidate merchant-cache for a particular merchant when saving/updating said merchant (which should mean less lag for updates to show on the front-end).
Remove banner for Vultr.com, which was displayed for relevant queries (like “VPS”), as we've had at least one unpleasant experience with their service.
Make a handful of small improvements to our administration tools.

January 2020

Continue to work on our outage-detection layer.
Make more tweaks/improvements to our product-detection engine.
Make additional improvements to our product-identification engine, with attention for iPhones and MacBooks in particular.
Continue to develop and debug our prospecting tools.
Begin work on new email-outreach tool.
Extend our blogging platform a bit.
Continue to develop aforementioned server-migration scripts.

December 2019

Make a number of tweaks/additions to our spam-detection strategies.
Spearhead effort to detect and persist advanced/generic “attributes” for products (where available), to be used to help us better identify products, among other things.
Rework the way we match/index “keywords” that have an alphabetic and numeric component (e.g., PS4, 5G) to avoid spurious results (leading to some performance improvements as well as the matching improvements).
Make a number of improvements and bug-fixes to our prospecting and on-boarding tools.
Extend our dead-product detection capabilities (in particular adding support for a strange case sometimes seen for old Newegg products).
Fix a bug related to the way we determine which products are due for checkup.
Perform substantial refactoring and reworking of the way we handle “background product-cleanup” based on incoming queries, with the notable differences being, making the process somewhat more efficient but also more likely to check up on products when we are not under significant load (with regard to background tasks).
Refactor the way we queue-up “background tasks”, some aspects of our merchant and subscription models, and logic for “opportunistically” checking up on products (amongst other things).
Add tweak for Overstock.com to avoid pointing users to the “featured product” pages that they provide us in data-dumps, as these pages can sometimes incorrectly show a product as being out-of-stock (due to some variants being unavailable, apparently).
Continue work on outage-detection.
Make routine improvements to our product-detection and product-identification engines.
Extend our “live tests” suite (which make various sanity and performance checks against our production website).
Work on some scripts to ease server-migration tasks.

November 2019

Sorry, we've not yet had time to compile our updates for October and November!

September 2019

Upgrade to newer version of database library (Slick, v3).
Begin implementing back-end support for product branding information.
Add tweak to better display names for gift cards (in cases where the merchant only includes the brand/store name but nothing indicating the product is a gift card/voucher).
Continue to improve our spam-detection capabilities.
Fix importation of tags/categories for some products in CJ-based data-dumps (which do not even keep fields in their correct columns).
Continue to improve our product-identification engine, in particular adding improvements for our ability to deal with compound words in cases where the single-word permutation (e.g., “doormat”) may be used in one place (such as in the product's name) but the multi-word permutation (“door mat”) used elsewhere (such as in product tags).
Continue to improve product-detection capabilites.
Make a minor bug-fix to one of our back-end product utilities.
Add special logic for purging Newegg-hosted products that are not sold by Newegg (Newegg “Marketplace” products, which are not available for BTC).

August 2019

Implement support for new data-dump format for importing products from Overstock.com.
Make some tweaks to our custom Newegg scraper/logic, to better crawl their site in line with recent changes they've made.
Improve support for dealing with abbreviations in search-queries and/or product names (e.g., a search for “doctor delight” should match on product named “Some Thing by Dr. Delight”).
Add beginings of some logic for detecting when a website has blocked our crawler.
Make some more improvements to our price-range-filter calculation code.
Continue to improve our product-detection capabilities.
Make more improvements to our spam-detection engine.
Extend testing over our on-boarding process.

July 2019

Begin work on making the amounts in our price-range filters “smoother” (with more to come).
Perform a substantial refactoring of code related to currency-conversion, making all access to our CurrencyConverter objects functional (making for more robust testing and easier future code changes in this department).
Fix bug whereby price-filter options would in some cases not be displayed if a price-range filter had been applied already and very few products resulted.
Improve UX for filtering by a custom price-range a tad, by auto-unchecking any selected ‘stock’ price-range options when entering the custom range.
Continue to improve our product-identification engine, with improvements related to avoiding descriptor details tacked onto a product name and avoiding products with really low-quality names (e.g., that have likely typos) altogether.
Add the beginnings of a system for determining when a site has blocked our crawler (which will eventually be used to respond accordingly, by adjusting crawl-rate, etc).
Extend some back-end tools related to merchant sign-ups/top-ups to make us a little more efficient.
Make additional improvements to our spam-detection engine.
Fix a little bug in the Onboarding Wizard.
Tweak styling for products qualifying for a Bitcoin discount.
Perform a number of additional code refactoring/cleanup jobs.

June 2019

Make a number of improvements to our product-detection engine.
Improve our ability to avoid false-positives when trying to determine whether a product is out-of-stock (e.g., by better avoiding text that indicates a “related” product is “out of stock”).
Add additional strategies to our spam-detection code.

May 2019

Fix our sitemap setup to properly use a "sitemap index", full of sub-sitemaps, which should (fingers crossed) lead to Google (etc) properly finding more of our search-result pages.
Rework “top-up” functionality to add beginnings of support for multiple payment methods (previously was on-chain BTC payments only).
In line with aforementioned, add support for Lightning Network payments for top-up.
Fix rendering of our reset-password screen, to make it look decent in the case where the user is authenticated.
Fixed links for Alza products (which was necessary due to some change or breakage on their end).
Make a few more tweaks to spam-detection.
Rework our code for pulling out “popular searches” a bit.
Rework a number of test-cases to avoid spurious failures.

April 2019

Make ongoing improvements to our product-detection engine.
Attempt to improve product relevancy by giving a slightly lower rank to products with a very undescriptive (short) name.
Continue to improve our spam-detection capabilities.
Fix a number of issues with our test-suite, which were leading to spurious failures.
Update some of the copy on our Bitcoin FAQ, in light of Bitcoin's older age.

March 2019

Add various <link/> and <meta/> tags to our site header to improve our mobile experience (e.g., “Apple touch” icon).
Improve our ability for avoiding false-positives when attempting to determine whether or not a product is still available (still in-stock), particularly with regard to the case where some “Related Products” are sold-out.
Tweak our name-normalization algorithm to better avoid junk like “Buy [thing]” being displayed.
Fix bug that was preventing some merchants from being properly enabled after completing the on-boarding process.
Add additional tweaks/strategies to our spam-detection filter.
Make routine improvements to our product-identification engine.
Fix another bug or two in our administration panel.
Improve/fix a number of things with our web-browsing test tools, in particular in the department of form submission.
Add more detail to our logs for POST requests.
Further consolidate our JSON-library use, to cut down on excess baggage.
Upgrade to latest version of scala-uri library (which turned out to require a lot more work than anticipated).
Tweak design of filter-panel for case where all matches are for one merchant.

February 2019

Implement a proper sitemap (finally), which will hopefully aid Google and friends in indexing a broader catalog of “things you can buy with Bitcoin”.
Improve relevancy-sorting, in particular with regard to “acronym nouns”, such as “TV” or “SSD”. This makes results for television much better, for example.
Make routine improvements to our product-detection engine, including (but not limited to) support for importing gift cards from Bitrefill.
Fix a bug whereby our Onboarding Wizard would crash if a merchant had already previously begun the on-boarding process and was starting anew (e.g., because something went wrong the previous time).
Change some of the sample searches offered on our homepage, providing queries that tend to yield more interesting / better results.
Implement “maxQueueSize” for our crawler, as stop-gap measure to prevent some large websites from causing memory-consumption problems.
Fix the subject-line for our renewal-notification emails to indicate the proper number of days or hours before a merchant's subscription will lapse (e.g., say “expires in 7 days” rather than “6 days”), and add testing related to these renewal emails.
Fix issue with Onboarding Wizard, whereby after activating a merchant that has chosen the Free plan, we invalidate the cache for that merchant to avoid problems when we first try to perform a full scrape/import.
Refactor our API to use a different library (than we were using) for JSON generation, allowing us to consolidate and cut loose a dependency.
Refactor our spam-detection code (structuring it properly into concrete “strategies”, making it easier to extend and debug), and continue to make improvements therein.
Fix a little bug in our “top-up” tool for merchants.
Make a number of bug-fixes and improvements to our back-end administration tools.

January 2019

Undertake beginnings of effort to reach out to Bitcoin-accepting merchants and encourage them to get listed in our index.
Fix bug in Merchant Suite tool to re-import products whereby the products weren't being properly imported/saved when performing merchant-initiated import.
Refactor some stuff related to our administration panel.
Refactor and extend some of the testing over our administration panel.
Make minor UI fixes to Merchant Suite Plans page.
Make some small improvements to our test infrastructure for testing at the website/web-app level.
Make another minor tweak to test-suite to better avoid at least one spurious failure.
Make more tweaks/improvements to our spam-detection algorithm.
Add spam-detection/prevention to our Merchant Suite lead form.
Fix a handful of bugs with our admin panel (introduced with recent reorganization over there).
Add hook to log a warning if we find a cache entry that is more than about three hours past its expiration, which probably indicates we are no longer keeping up with maintaining a fresh cache — which may be indicative of a broader problem, like the issue with the hosting/hardware we experienced this month (which nearly brought us to a halt, and led to queries taking about 10x what they ordinarily would, maybe more).

December 2018

Improve the way we deal with third-party “placeholder” images (e.g., stock images that say “no image available” and the like).
Make some (small) improvements to product-identification engine (with some particular attention to results for “samsung galaxy”).
Make some improvements to our product-detection engine.
Extend our admin tool for viewing details about a particular product.
Fix a bug related to deleting thumbnailed images, which was leading to some products being rendered with broken image references.
Make some minor improvements to our spam-detection tool.

November 2018

Migrate to a new server (which, so far, seems to perform better!).Create admin-tool to help us extend merchant subscriptions manually, when necessary.
Reorganize our administration landing-page, making it a bit more pleasant/easy to use.
Continue to improve our product-identification engine, as always, with some particular attention to classifying t-shirts and “stockings” better (for example).
Tweak exception-handling/logging with regard to our background product-checking, to make it easier to see what product(s) are culprit when there are issues.
Fix a minor bug in the caching layer, with regard to determining whether or not we calculate price-ranges for low-result queries.
Redesign the “highlight” styling applied to products that qualify for Spendabit Discounts (i.e., products that are available at a discount when purchased with Bitcoin).
Patch little ‘vulnerability’ with the mechanism we use to serve our CSS.
Add some additional exception-handling to BTC price-lookup hook.
Change our “og:image”, leading to our logo showing up much better on the thumbnails used in recent versions of Firefox and hopefully in various social networks as well.
Refactor some of our tests a bit, to better avoid spurious failures.
Continue to improve our product-detection engine (business as usual).

October 2018

Add beginnings of a “long-term caching” system, to be used (for example) to store calculated values that will rarely (if ever) change.
Rework our back-end tool for viewing application ‘threads’ to avoid problems when/if we cannot find a stack-trace for a given thread (presumably because it has died out during the process of gathering thread info — a race-condition of sorts).
Perform a refactoring of search back-end to further streamline the way we pass around/back search details.
Begin retaining information about the execution time of each query we process, which we will soon use to monitor our search performance.
Fix little bug with filtering for our forthcoming “alts” support.
Implement the beginnings of a better database-migration system, to ease development headaches and make deployments (in some cases) easier.
Work on implementing password-reset tool for subscriber merchants (half complete).
Refactor a few UI-related details.
Refactor our configuration-bootstrapping code a bit, making it easier to test some use-cases.
Enforce a character-limit on prices (for product filtering), avoiding crash in (rare) cases where someone enters a crazy number with like 300-some nines (“99999999999...”).
Upgrade our database connection-pooling library (HikariCP).
Add more tweaks/hacks to avoid saving gift-cards sold at Newegg.com, as those are not available for purchase with crypto-currency.

September 2018

Perform considerable refactoring related to the way we pass around application “services” (e.g., HTTP client, email-sender), improving the way we “stub out” when testing, and improving the code's modularity in general.
Further improve our <meta/>-tag page descriptions.
Improve integration of Business Directory into search, a tad.
Add some error-handling to our cache-updating mechanism, to avoid spurious errors/noise in our logs, as well as failed cache-update attempts, when relevant product happens to get deleted while updating cache.
Improve our forms library to support file-upload fields.
Create admin tool (leveraging aforementioned improvement) for uploading arbitrary images to our DB (so we can manually update a merchant's logo, for example, more easily).
Refactor some things in our admin portal.
Tweak the way we render our product grid a tad, for some device/screen sizes.
Make a number of improvements to our test suite, to better avoid spurious failures and such.
Make some other tweaks to our test suite, aiming at eventually having it run much faster (by further taking advantage of parallel processing / multiple CPU cores).
Continue to improve our product-detection engine (as always).

August 2018

Implement “alt-coins form”, where merchants can indicate which crypto-coins/currencies they accept.
Extend aforementioned form to also work for visitors/merchants that are not yet listed on Spendabit (in which case it will allow them to enter their URL, email address, etc).
Make more improvements to our <meta/>-tag descriptions for search-result pages.
Do some more work to filter-out/purge gift cards sold at Newegg from our search-results, as Newegg does not allow customers to pay for gift cards using Bitcoin (unfortunately).
Begin work on an “outage-detection” sub-system for keeping track of problems at third-party (e.g., merchant) websites.
Improve administration capabilities for troubleshooting client/merchant issues.
Make some minor UI improvements to merchant ‘panel’.
Fix (not too common) crasher that could occur if someone uploaded an invalid or unsupported image-file in the Onboarding Wizard.
Tighten up wording a bit on ‘Choose Plan’ page of Business Directory path of Onboarding Wizard.
Tweak our spam-detection mechanism a tad.
Rework some things related to logging on the back-end.
Continue to improve our product-detection engine (como siempre).

July 2018

Make some changes to our caching layer, and add further limits to pagination (now only up to 1,000 products may be seen by paginating), cutting down on slow queries (most of which were being generated by bots, presumably).
Add support for identifying “dead” products in cases where the product's URL yields a blank page/response (no content), but the response status-code is 200.
Fix bug/crasher in Onboarding Wizard that would occur if user entered a bio of approximately 260 characters (which was below are form-validation limit but just above the allowed length for the underlying DB field).
Add a “timeout” to product-import process of Onboarding Wizard.
Continue to improve our product-identification engine, including more improvements to the way we determine what is “color”-related information and some specialized logic to better identify smart-phones.
Continue to improve our product-detection engine, including improvements related to determining product name, description, price, and more.
Make minor improvements to some of our admin tools.

June 2018

Tweak some things on our homepage, aiming to cut down on the ugliness experienced on page-load in some browsers.
Make use of <h1> tags on search-result pages, in hopes of improving SEO.
Fix crash that could occur with Onboarding Wizard if client used browser back-button after attempting to import products.
Aggregate all of our CSS into one file, for any given page, so web-browsers need only make one HTTP request to for our CSS, hopefully leading to a better/faster rendering experience in most cases.
Further improve Purse.io integration, making the presentation much nicer and offering the link to users when they're on the last page of results (in addition to displaying it for no-result queries).
Properly store product descriptions, where available and when we successfully extract them (which will probably soon be used to give more product detail to users and is already used to better identify products).
Make a bug-fix to our Bitcoin-invoicing tool (presently just used when subscribers top-up or make a subscription) with regard to the way we render the BTC amount.
Refactor the way we parse and store incoming user queries/searches (before we send them off to the database).
Do not attempt to scale already-small images when generating thumbnails for products, avoiding excess noise in our server logs if nothing else.
Continue to improve our product-identification engine, giving particular attention to dealing with products that mention a geographic region (state, province, country, etc) in their name, as well as products that use “coffee” or “chocolate” as a color as opposed to a noun (describing what a product is).
Continue to improve our product-detection engine (as always); in particular, improving support for pulling out price for some products that have a range of prices (amongst other improvements).
Beef up our spam-prevention code a bit more.
Fix styling issue with our error page (which, luckily, is not seen too often) that we accidentally introduced when doing work to aggregate our CSS to be served in one request/file.
Remove some unnecessary AWS dependencies (as we only need S3 support).
Make some minor changes to our Feedback/Contact page.

May 2018

Avoid showing not-yet-activated merchants on Browse by Merchant page.
Refactor the way we deal with hyphenated “keywords” to be less hackish and avoid some issues with our caching layer.
Further integrate Business Directory listings into search.
Avoid retrying requests (e.g., for product pages) when response has no body/content but the Content-Length is explicitly specified as 0.
Tweak the way we calculate price-ranges in some edge cases.
Implement better page descriptions (i.e., <meta name=description />) for Google and other search-engines to index, for search-result pages.
Offer users a link to Purse.io when we have no matching results.
Better avoid crashes on our Search Partners registration page.
Fix issue whereby a product whose page yielded a 40x response would be considered “dead” even if the site's homepage yielded a 40x response (when such a state of affairs indicates a broader, and not-product-specific, problem).
Adjust copy on Popular Searches page.
Make some improvements to our product-detection engine.
Make some improvements to our product-identification engine (adding somewhat improved support for dealing with “compound-word” products, like pillow case / pillowcase).
Further tweak our product-identification engine to better deal with the difference between “accessories” and the things they accessorize.
Be more particular when deciding how relevant merchant-provided keywords are in identifying products (preventing some desktop computers from showing up in searches for “laptop”, as one example).
Remove Expedia from special banner for “hotel” searches, as they have apparently halted Bitcoin acceptance.
Make tweaks to avoid listing Newegg-listed gift cards, as Newegg does not seem to allow you to pay in BTC when purchasing gift cards.
Add the beginnings of performance test-coverage over product-detection engine.
Improve some back-end logging stuff.
Implement additional spam-prevention measures.
Refactor some of our sorting-related search code.

April 2018

Fix a couple issues/bugs with our Bitcoin invoicing tool (at present just used by subscribers when signing up or adding credit), refactoring invoice-creation to be more testable at the same time.
Begin adding test coverage over our invoicing tool.
Continue to improve our Onboarding Wizard, including a bug-fix to allow clients to use browser 'Back' button after having already approved their scraped products.
Begin presenting Premium Business Directory subscribers in our Featured Merchants lineup.
Begin integrating Directory businesses into our search-results page, where relevant.
Implement UI support for merchants that are BTC “Exclusive” (products you can only buy with Bitcoin).
Change the way we calculate price-ranges for some (single-word) queries, which dramatically improves the usefulness of the price-range “buckets” offered (for example, when searching for iPhone).
Improve our color-detection capabilities a bit more.
Normalize plural forms of brand-name “nouns” like “iPhones” properly (at least for those brands/nouns that are known to our dictionary).
Properly use custom scraper/importer (once again) for merchants that have one.
Begin building system of standardized “product sets” for our software test-suite, which should make adding new test-cases easier in some cases and (eventually) be leveraged to make our test-suite run faster.
Finish up presentable version of a blog-post giving a high-level view of modern HD Bitcoin wallets.
Make many tweaks/improvements to our product-detection (scraping) engine.

March 2018

Continue to make progress on our Onboarding Wizard, and begin directing a portion of our “Get Listed” traffic there.
Change a database setting, after many hours of research and experimentation, on one of our central indexes, which fixes an annoying issue whereby some product UPDATEs (and some INSERTs) were taking an exceptionally disproportionate (long) amount of time to execute. (For those interested in the intimate details of PostgreSQL, we turned off the FASTUPDATE index setting.)
Avoid crashes when we receive POST requests without proper parameters (usually intrusion-detection bots or the like).
Continue to improve our product-identification engine, including (in particular) tweaks for “music” (like albums) and computer storage drives.
Improve our back-end tools for viewing product details (used for debugging).
Don't display “special banners” (such as that displayed for searches for hotels) when filters are enabled.
Avoid crashes in some rare cases where client gives a ridculously-long User-Agent string.
Refactor the way we fire up new “scrapes”, so as to better track when/how particular products were added or updated, and provide a more efficient mechanism to clear old products post-scrape.
Improve support for displaying discount-code for merchants that offer a Bitcoin discount.
Improve searches for HDD.
Begin refactoring the way we deal with “services” for interacting with the outside world (e.g., email, HTTP) under the hood, to make for easier / more-elegant testing in the future.
Fix bug whereby we were deleting legitimate products after a fresh “scrape” for some merchants (which was due to a change whereby we estimate the number of products updated, to improve efficiency).
Upgrade to latest minor version of web framework and related libraries.

February 2018

Enable use of our caching layer for queries that have merchant and/or price filtering turned on, speeding up some such searches significantly.
Continue to make progress on our Onboarding Wizard...
Add some internal/automated tests over said Wizard (with much more still needed).
Add beginnings of support for capturing product descriptions (which may in the near future be displayed in something like a “product tooltip”, amongst other potential uses).
Add beginnings of support for using product description to better identify products.
Improve color-detection capabilities (used when identifying products, but not quite yet for things like filtering by color).
Improve product-detection for sports-related items a bit.
Make other improvements to our product-detection engine (as always).
Tweak our query-parsing engine to, for example, make searches for "10 oz" and "10oz" roughly synonymous.
Tune our relevance engine by adding a strategy whereby we prefer products that contain the full search-phrase (query), in cases of multi-word queries.
Properly set Content-Type when putting image-files in S3 bucket.
Improve support for finding and storing merchant favicons, in particular in cases where the image is not in PNG nor JPEG format.
Improve the way we deal with product-pages that give us back a 500 (error) response-code.
Automate compilation of our Featured Merchant lineup.
Adjust CSS on homepage a tad, to make things render better on small devices/viewports.
Add an alert to the last page of results for filter-enabled, cache-based searches, suggesting to “refine your search”, as in some such cases, there may be matches that don't show due to the limit on the number of products we'll cache for any given query/keywords.
Display a more sensible blurb on merchant-profile page (below merchant bio and logo) when merchant has no products.
Fine-tune other odds and ends to make the system run more smoothly in general.
Put together blog-post with some of our thoughts on Bitcoin Cash.

January 2018

Make more progress on our Onboarding Wizard (getting there).
Add beginnings of documentation on “import compliance”.
Improve support for dead product detection, in cases where product page yields a 500 response-code (whilst trying to avoid purging legitimate products hosted on a website with a temporary outage or problem).
Better handle exceptions on our embedded search page (one of the options we offer Search Partners.
Put in place additional measures to reject “junk” queries, to cut down on useless resource usage.
Avoid slow COUNT database-queries when refreshing the search-cache for some large result-sets, to avoid needless resource usage.
Change the way we display some product images, making them render much better in such cases.
Make tweaks to better support searches for t-shirts.
Improve our internal tools for inspecting product details (which aids in debugging and fine-tuning our search relevance and such).
Add support for using our caching layer to execute queries that have price-filtering enabled (thus speeding up such queries, in most cases).
Add to our “live tests” suite, which aims to provide another level of quality control (above and beyond our general test suite).
Improve support for differentiating between “accessories” for a particular type of product (e.g., “iPhone case”) and the actual product (an actual iPhone).
Add support for SVG images to our Merchant Suite Dashboard (allowing subscribers to upload their logo in SVG).
Improve support for detecting “discontinued” products.
Improve support for detecting defunct websites, in particular when there are issues with the SSL/security certificate.
Avoid database issues in cases where we pull out too much “tag” data (e.g., product categories) for a product.
Upgrade to a more recent version of jQuery.
Make more improvements to our product-detection (scraping) engine, of course.

December 2017

Display discount codes in tooltip of “discount ribbon” for relevant merchants.
Begin work on a blog-post about how HD walles work.
Add support for caching SVG images.
Make some improvements to our error-reporting mechanism, to better aid us in debugging problems.
Add better validation to our Search Partners signup form.
Start caching thumbnails for all products (previously was only those with non-HTTPS URLs).
Add support for additional ID format at Newegg, which should result in a few more relevant products getting indexed.
Minor improvements/tweaking to our ability to detect “computers”, and similar products.
Merchant Suite Panel: Add “Featured Merchant Clicks” section to Stats page (for Enterprise merchants).
Switch rotating blurb on homepage to be for queries that yield products with Bitcoin discounts.
Take further steps to better support merchants which we cannot scrape (due to JavaScript-reliant websites, servers rejecting requests, etc.) by allowing for products to be imported via CSV.
Make minor tweaks to our query-parsing code, to avoid some unnecessary search-cache updates.
Make some tweaks to our background “product-checkup” process to make it a little more efficient.
Add new 'attributes' field to allow each product to have arbitrary, semi-structured product metadata; to be used eventually to provide advanced product filtering, as well as to improve our product-identification engine for better search relevance/sorting.
Make a few tweaks to our Buy Bitcoin page.
Purge a couple unused columns from our main products table (database), as well as an unused index.Upgrade to latest Scala minor version.
Upgrade to latest version of web framework we leverage for routing requests, rendering HTTP responses, etc.
Upgrade several other dependencies.
Continue to make tweaks to our log-monitoring infrastructure/configuration, which helps us to better weed out the noise in keeping the system's “pulse”.

November 2017

Implement Spendabit Discounts.
Make more progress on our Onboarding Wizard (still not quite ready for prime-time at this point).
Swap out back-end service used for currency conversions (again).
Make all currency exchange-rate refresh lookups executed as background tasks (using cached rate for all requests, until background update completes).
Tweak our search-query parsing algorithm to avoid excessive/spurious results for queries with hyphenated “words” (e.g., make a search for “quad-core” not show results that only have “quad” or “core”).
Make small UX improvements to search-filter options (at least for some versions of Firefox).
Begin using sitemaps to guide our crawling (when looking for products).
Improve robustness of our "cultivation" process a bit.
Improve robustness of our thumbnailing mechanism.
Avoid effectively duplicated querys in Popular Searches, by normalizing plural-vs-singular to deduplicate.
Make tweak that should have background-task threads run with a lower priority, thus (in theory) improving performance of threads that serve user requests.
Expand the number of products we cache for any given query to better avoid cases where we run out of products before max pagination is hit (due to products becoming out-of-stock or defunct between cache updates).
Make minor tweaks to our blog software/code.
Continue making improvements to our product-detection and product-identification engines (as always).

October 2017

Display Merchant Suite subscribers on Directory page.
Improve our product-identification code for detecting computers (PCs).
Make minor tweaks to homepage.
Make some long-needed improvements to our product-grid rendering on small viewports/devices.
Make search-input placeholder text shorter when on really small viewport (at initialization), so it doesn't overflow the input.
Improve renewal emails a bit.
Do many other things that we forgot to document.

September 2017

Begin work on rehaul of our homepage (with more to come), aiming for better SEO juice as well as better usability (so it's clearer what Spendabit is when people stumble across it).
Allow for filtering products on mobile devices / smaller viewports (finally!).
Continue progress on our forthcoming Onboarding Wizard.
Tweak the way we throttle requests when crawling the Web (for products) to avoid pausing for longer than necessary.
Tweak custom-price filter to look better on some viewports (those just larger that 'xs').
Rework way we send emails (to run as background activity) to avoid tying up request-handling threads (which seemed to be lending itself to occasional site outages).
Finish v1 of “top up” tool for Merchant Suite subscribers.
Improve/tweak styling of invoices.
Implement <link rel=next /> and <link rel=prev /> for pagination of search results (which should help search-engines better understand our site structure and hopefully improve SEO).
Make some improvements to how/when we thumbnail/cache product images.
Only show special “banners” (e.g., for BitRefill) when displaying the first page of search results.
Increase allowed length for product URLs, meaning fewer products will be rejected due to their URLs being too long.
Improve ability to track down (correct) favicon for merchants by avoiding spurious files/images at default favicon location (/favicon.ico) that are too small, such as useless 1x1-pixel gif images.
Lots of tweaking of our sales/lead pages, with most attention given to the Search Partners page, primarily aimed at making them look and work better on small devices/viewports.
Make some tweaks to how we “round” and display result totals.
Make a number of SEO-related changes.
Continue making improvements to our product-identification engine (as always).
Continue making improvements to our product-detection (“scraping”) engine (as always).

August 2017

Make a significant refactoring to some of our core search-related code — which by itself will be unnoticable to users, but lays the groundwork for some long-needed improvements...
Add ability to set a custom currency (including BTC) for price display — finally!
Implement ability to filter by a custom price range — finally!
Further enhance our product-identification engine by adding additional pattern that we look for in product “tags”.
Improve matching for products with domain-names in their name — more specifically, a search for amazon gift card will now match on products with “Amazon.com” in their name.
Improve support for discerning product color from product type.
Fix keyword-normalization for some edge-case words (“drive”, in particular), making matches for things like “1tb drives” better.
Make some improvements to our code for detecting defunct merchant websites.
Do some work that should pave the way to allowing merchants to import their products via CSV (for cases of non-compliant sites).
Improve (slightly) our internal process/machinery for dealing with incoming feedback/inquiries.
Make minor improvements to Merchant Suite Dashboard.
Rework some of our English dictionary stuff, and make further improvements/tweaks to our product-identification engine.
Add support for caching (to S3) images that our underlying image library cannot parse but look pretty clearly to be valid JPEG files.
Kick off work on our forthcoming Directory service.

July 2017

Fix bug that was preventing auto-downloaded favicons from being displayed.
Add new Buy Bitcoin page, and point users to it when they search for something like “bitcoin”.
Extend search “alerts” to support region-specific queries.
Setup the logcheck utility to keep a closer eye on our log files, so we can have a better “pulse” on our server/application.
Make some improvements in our ability to detect wether a product is in-stock vs. out-of-stock.
Finish implementing mechanism for auto-downloading and saving favicons for each merchant.
Add new mechanism by which we can “proxy” thumbnails for products whose images are hosted at non-HTTPS URLs and for which we do not already have a thumbnail generated (and stored on S3), allowing us to avoid pages with “mixed content” (resources served via plain HTTP) in some cases.
Make improvements in our ability to reject some useless queries (e.g., 1), to better cut down on unnecessary load on our database.
Make more improvements to our product-identification engine (which in turn makes searches better at giving back relevant results), with particular attention to avoiding confusing accessories as being the things they accessorize.
Add mechanism for checking up on products presently flagged out-of-stock, to check they haven't come back in stock.
Remember entered email address if login attempt fails.
Minor UI tweaks to header for mobile-sized devices/viewports.
Improve our ability to find favicons at merchant websites.
Improve validation over our Merchant Suite form.

June 2017

Write blog-post about Bitcoin volatility.
Work on mechanism for auto-acquiring merchant “favicons”.
Rework the way we clean out old products after a fresh import for a particular merchant, to avoid attempting to load-in too many products all at once for merchants with a really large number of products — which appears to have even been a source of occasional site outages.
Rework our caching layer to yield much better results for some multi-word queries. For example, previously searching for water bottle would yield roughly the same results as bottled water. Now, the results are unique and as you would expect.
Make some improvements to detecting product “tags”, which are used to better identify products and thus make search-results better.
More improvements to our general product-analysis engine, also to help improve search-results.
Minor tweak to our homepage to make the tagline render better on certain (smaller) device sizes.
Some changes to the way we're serving “static” assets, including adding use of HTTP-level compression, which should make our site slightly snappier.
Give proper 404 response for requests for non-existent merchant profiles.
Do some behind-the-scenes work to try to figure out why our app-server occasionally runs out of memory.
Disable a number of defunct merchants.

May 2017

Add this page.
Tweak to better support some Newegg products. In particular, some Newegg products are sold both as “Marketplace” products — products sold by third-party merchants via Newegg.com, which are not available for purchase with Bitcoin — as well as directly via Newegg itself. With this change, the user should be presented with the Newegg version of the product (which can be purchased with Bitcoin) by default.
Allow for searches that are purely numeric (e.g., 360, which might return results related to Xbox 360, amongst other things), provoked by at least one real-world use-case/complaint. (Previously, such requests were being rejected to avoid unnecessary load on our database, and we still will reject one-digit queries (e.g., 5), as it's hard to think of a real-world use for such a query, and they tend to have large result-sets.)
Update list of sample searches (displayed under search-bar after a few seconds if you don't enter a query).
Add placeholder text to search-input, to make it more clear Spendabit searches, and that it searches across many merchant websites.
Fix to make our web-crawler more “polite” (again).
Some improvements to make our web-crawler a bit more resilient to error (e.g., in cases where a website has a broken redirect mechanism).
Improvements to our ability to detect “remote IDs” for products.
Scraper improvements to support AlzaShop.com.
Some work on tool to allow merchants to “top up” their Merchant Suite subscription.

April 2017

Couple of CSS tweaks to avoid “jumpy” elements (e.g., products) on page-load.
Couple of minor UI tweaks/improvements to Merchant Suite Dashboard.
A couple of (mostly CSS) tweaks to make the Merchant Suite and Search Partners sales pages render better across different devices / viewport sizes (though there's still a good bit of work that remains to be done here).
Search improvements for: “hoodie(s)”, distinguishing between “music” (e.g., CD albums) and musical instruments, distinguishing between “accessories” and the items they “accessorize” (e.g., distinguishing between an iPhone and an iPhone case).
Remove a couple of defunct merchants.
Write blog-post/comments detailing our 2¢ on the ongoing “scaling debate” / drama.
Fix bug with Search Partners signup form.
Add Bitcoin “price ticker” to homepage.
More scraping (product importation) improvements, as always.
Avoid crash (due to annoying dictionary library we use) in cases where someone enters large nonsense “word”, like thisistotallybogusjunk.

Unfortunately we did not begin tabulating changes and improvements here prior to the above-listed items.

July 2014

Spendabit launches!