Control Your Data: a Disco and The Indexing Company Case Study

Crypto has the unique characteristic of putting the user back in control of their own data, which enables the opportunity for data to be made available wherever the user goes. This context - in which the subject of data has the most control over its data - is often described as Self-Sovereign Identity. 

The user determines their on-chain identity, controlling which data and assets can be provided to other parties, alongside off-chain data created throughout their digital journeys. Managing, and even viewing, all this data from various chains and digital platforms can be a tedious task. Identity protocols like Disco have the vision that users can consolidate their online presence into a logically centralized, physically decentralized data package, controlled with their existing wallet. 

Data indexing and methodology is critical to this identity layer. Most businesses in crypto struggle with accessing and organizing complex multi-chain data, spending valuable time and resources on building indexers, data engineering and constructing data pipelines. The Indexing Company is of the opinion that crypto businesses would be able to provide more value to users when focusing on their expertise and not worrying about indexing and its associated challenges. In our collaboration with Disco, they could fully focus on building the future of seamless access, instant rewards and all of the other fun parts of the on-chain world enabled by our unique identities. With Disco, over a million users had a product where they could have control over their data and have privacy when preferred. To expand the utility for users to express their online identity, Disco was acquired by Privado.ID. The complementary teams join forces to build an identity network providing next level utility for users, enterprises and governments. The Indexing Company is happy to support this newly formed team in their future ventures. 

In this case study we will take a look under the hood and show you which data and technology by The Indexing Company provided on-chain data and context to the Disco Data Backpack and Disco platform.

The Setup

The process started by understanding which types of data are the most important to users. For example, Disco used specific data from chains like Ethereum, Arbitrum, Optimism, & Base, like significant tokens and NFTs. The dataset scaled to accommodate popular queries over time. The Indexing Company’s unique data pipelines can fetch data from any chain while also making it possible to fit this data into a single unified data schema. Such transformations helped Disco with building a data model from which they easily expanded the number of chains or on-chain assets indexed. The data was directly streamed from the RPCs to Disco’s database reducing latency and ensuring data quality for both Disco and the end user. 

The Indexing Company gives projects working with the data pipelines ownership over their data. Since the public RPC data was streamed directly to Disco’s backend all data was fully composable for application in products and redistribution in numerous forms. 

Just In Time Indexing

Disco's user base continuously expanded as more crypto users recognized the value of their own data. This growth presented challenges in data retrieval, as such a system must integrate historical data for new users while simultaneously accommodating incoming data. Historically, projects had to index entire chains, incurring high processing and storage costs, despite only a small fraction of the data being relevant to their users. Drawing from experience, The Indexing Company recognized the impact of this industry-wide problem and developed Just In Time Indexing (JITI). JITI enables getting both historical data and real-time data whenever the data is needed. For Disco, this involved invoking the indexer with a wallet address, which triggered JITI to perform a targeted backfill of all historical transactions. It also ensures data remains current by monitoring new transactions for those wallet addresses in real time. All chain and top token/NFT holdings were served to current and new users whenever they wanted to use Disco. At the peak of our collaboration, Indexing Co. processed data from more than 1.2 million wallets for Disco using JITI. 

Contextual data

Without context, on-chain data is hard to access, interpret and use. On-chain data consists mostly of numbers and contract addresses which do not tell you basic information on the protocol or user. For users, this is a positive feature creating pseudo-anonymity, but for interpreting data around which protocols are being used, it creates a problem for analysts who are trying to create valuable insights for their business and users. For Disco, understanding the platforms and actions users engaged with enriched the context in their Data Backpack. Disco users had the option to carry this valuable information with them to new platforms and to expose which data they wanted. The Indexing Company is continuously grabbing labels from various data sources. First, various data platforms like Etherscan and Dune provide labels. Second, on-chain logic can be used to generate labels. For example token-meta data and factory contracts can be indexed to automatically label Uniswap pools. When this data is matched with on-chain activity we can see if a user is a Uniswap user and which types of tokens they swap. This logic can be applied to a variety of (DeFi) protocols to get a better understanding of transactions. Through an iterative process where analysts and developers from both Disco and The Indexing Company continued to collaborate, more labels and contextual information were gathered over time. 

Control your data

Disco and The Indexing Company found each other in the value that you should control your data. Disco users benefitted from taking control over their data through the Data Backpack, while the raw blockchain data was fully owned for free analysis by Disco themselves. The Indexing Company is happy to support builders like Disco in their journey to bring a better user experience in crypto. Our work enables businesses to focus on what actually matters: their product. We wish the Disco and Privado.ID team good luck in the next iteration of building their vision: a complete multi-chain, multi-device identity protocol for every user.

Get in touch with us to see how we can help build your data infrastructure so you can focus on building user-centric products. Your data, your way.