Page 175 - Special Topic Session (STS) - Volume 3

STS538 Pedro Luis do N. S. et al.

Figure 1: Experimental weekly price variation derived for airfares, with
series derived for data collected by manual and automatic tools in the same
dates. Reproduced with permission from [da Silva et al., 2019].
Although the access issues and the requirements for maintenance of the
web scraper are important limitations to have in mind, a series of benefits can
be derived by the adoption of the automatic process. The time and effort
required for data capture are substantially reduced, hence freeing time the
price collectors can allocate to other tasks. The automatic process enables
creation of high-quality data backups, since snapshots are automatically
extracted from the screens where the prices were collected, enabling
verification of the information collected. Finally, the automatic process also
enables increasing the number of price quotes for routes in a straightforward
and cheap manner, which in turn may lead to more accurate estimates of the
CPI.
In order to guarantee that the IBGE robots are not going to be blocked,
IBGE has initiated a conversation with the airline companies to negotiate
access agreements. This process is also fundamental to create a relationship
between IBGE and the companies, which may help to re-solve technical issues
that might arise, for example, when their websites are expected to change.
b. Transactions e-records
Many NSOs are devoting efforts towards the use of scanner data. Scanner
data from private retailers often contain very detailed information on the
products traded such as: prices and quantities of products sold, the time of
the purchase, etc. This source is more attractive than web scraping, because
prices recorded correspond to goods and services actually traded. However,
access to scanner data is much more difficult in the absence of legislation that
grants the NSO legal rights of access to such data, as is the case in Brazil at
present. Therefore, individual negotiations with each retailer are needed, and
in some cases, may take years until an agreement is reached, which may or
may not require the NSO to pay for the data to be provided [ILO, 2004].

164 |I S I W S C 2 0 1 9