APIs cover part of the data landscape, but by no means all of it. What if the data you need is locked on human-readable web pages or simply can’t be accessed efficiently.
Smart Feed provides fresh, highly scalable data from web pages and existing APIs as a single feed for your customers or internal intelligence.
We service a full-scale price and inventory feed for hotel data, but speak to us about any data accessibility need you have.
The web is the richest source of data on the internet.* But apart from a few disparate APIs, data resides on pages only accessible to human users. Prices, product information, key statistics and information about competitors is locked away from computer interpretation. Where APIs are available, data sources are fragmented and brittle meaning data hungry companies staff costly development teams to acquire it.
Smart Feed provides the single API for data on the web. Using robots to schedule and crawl any information – even if it is locked on web pages – results in a rich, fresh data view which can scale to cover billions of reliable data points. Smart Feed connects everything and provides a consolidated API so developers and Data Scientists can concentrate on the data and save money on engineering resources.
* Estimated to be approximately 7.38 petabytes of data in February 2022 (Source: https://www.worldwidewebsize.com/ x avg. 3Mb/page)
There is often no viable API connectivity for data available only on third-party websites.
Smart Feed can access any data on the web, gather it at scale, and bundle it together as a single feed.
A major Hotel Metasearch website
This Hotel Metasearch wanted to provide whole-of-market search results for hotel room rates and availability on a global scale.
However, it was only able to connect a small number of hotel chains with reliable data.
Smart Feed can access any website, turning it into a source of fresh, reliable data. This is perfect for hotels, and can provide a full market price and inventory feed.
With Smart Feed there are no connectivity gaps. If the source data is available on a website this can be provided in the feed. Data is either pushed on a predetermined schedule or can be pulled on demand from our API.
The Hotel Metasearch shared the data sources it needed. Hotel websites are generally grouped by booking engine, so this meant training the Smart Feed robots to gather the relevant rate and availability data from each booking engine, rapidly covering thousands of individual hotels.
Because rates change and rooms get booked all the time, the data needs to be fresh. Smart Feed deploys a set of higher-level automations which determine when the rates change and inventory numbers go up or down. These automations determine the optimal schedule that Smart Feed uses to crawl the underlying hotels and update the data.
The feed is regularly pushed to the Hotel Metasearch who perform accuracy checks and then display the data in their search results. Their users then rely on that data to select their hotel and go on to book.
If the Hotel Metasearch needs fresh data for a particular hotel, that’s possible too. They simply pull it using our API.
Smart Feed technology uses web automation. Each script loads the connected website in a regular Chrome browser. Data is then obtained and interpreted. All this operates on a massive scale – millions of automated interactions per month – so coverage is as broad as needed and the feed remains fresh.
Smart Feed can also access regular APIs. These, in combination with crawled data from connected websites, provide a single, simple feed for myriad data sources. The Hotel Metasearch has no need to maintain multiple, brittle connections to various sources and APIs itself. Smart Feed provides a single, consolidated feed for everything.
Smart Feed can gather data from any web source and package it as a feed. There are two, primary use cases:
1. Where limited data is available from APIs, but data is present on websites, Smart Feed can pull all of it together providing a complete data picture for a given subject.
2. Where development teams maintain multiple, unstable or disparate data sources – from APIs or from the web – Smart Feed offers a single, simple consolidation. This means no longer staffing dev teams to maintain messy and brittle data connections.
Smart feed can deliver data feeds at scale for hotel, travel, retail, grocery, logistics, financial services, BI, price intelligence and for statistical analysis. In fact, if the data is out there somewhere on the web, Smart Feed can supply it to you as a feed.
Our feeds conform to a consistent schema, and can be Pushed to you on schedule. Alternatively you can integrate against an API to pull data on demand.
We work with you to expand our data coverage, bringing on support for additional services without in-house Automation Team.
We deploy sophisticated scheduling techniques that ensure maximum data accuracy at minimal load on the underlying sources.
Our feeds are highly optimised, and run 24/7 ensuring you get a trustworthy data source no matter the time you access the feed.
Discover moreWe provision finely tuned infrastructure designed to operate at the scale demanded by our data partners.
Our resources scale on-demand, and deliver a high volume resource so data remains accurate and valuable no matter the underlying load.
We’ll work with you and your team to demo how powerful the Automation Cloud is and how it can benefit your business and use-case.
