Huge Update. Part 1 — The Tech
Normally we spend a lot of our time breaking down the complexity of data for the uninitiated because, at the core, TIKI wins by bringing the tools and techniques used by us data nerds to everyone, users and businesses alike.
This update is a bit different. I wanted to spend time celebrating the tech we've been creating. I use the word "create" because TIKI, at a most fundamental level, is a very different approach to data. Meaning we had to create new systems and patterns just to make it possible.
This is not the type of project where you bolt together some existing tech, call it "MVP," and start slinging data. That's not a knock on the amazing entrepreneurs who've built killer companies that way. It's just that solving the world's data problem requires something entirely new.
I'm going to present the tech at a 10,000 ft. view. As always, all of the code is on Github, but I'm also happy to share our far more technical design overviews/whitepapers if people think it would be helpful. Just leave a comment or drop me a note (mike@mytiki.com).
We've been hard at work on three core components of TIKI's platform, which represent what we call the "core infrastructure." At the end, I will touch on two additional components that we're starting to build.
1. The mobile app. Seems innocuous enough; who can't build a mobile app these days? True, until you go to build a decentralized, anonymous data engine. User privacy, anonymity, protection, and ultimately trust are central to our mission.
The only truly safe place for us to handle user data is at the edge, i.e., the phone in your pocket. This way, even we at TIKI can't ever see or access it. This is because your data on the internet, in the services you use, is not anonymous. If a server goes and fetches said data on your behalf, it knows who you are. Maybe not day 0, but eventually, it will be exploited.
In tech, we often call this an "anti-pattern." Typically mobile apps act more like a web browser than a server. Servers handle the hard work. Browsers and apps show you the results. We need to do the opposite; we need to move the hard work of a data pipeline down to the app.
This presented some interesting challenges we had to overcome.
Mobile developers are not data engineers, and data engineers are not mobile developers. Yes, this makes hiring tricky. But, what you don't see below the surface is that the vast majority of the wonderful SDKs, libraries, and software patterns the engineering community has created over the years are suddenly useless.
The best example is one our team directly experienced. We started out coding the app in a popular Flutter pattern called BLoC. Only to find out a couple of months in, after the code base grew sufficiently large, with many interrelated dependencies and data flowing, it just would not scale. So we modified our approach to a personal favorite of mine from microservices development: vertical slices, bringing along from BLoC the use of Providers. This worked well for a few months, but ultimately we hit another wall once the code base got so large that we spent all our time chasing bugs. Every new feature felt like one step forward, two steps back. So we took the lessons learned and adapted again. A pattern we call "microlibraries," blending the power of the mobile development framework, Flutter, with the vertical slice architecture of backend services. And…wow. It exceeded our expectations. We've been building faster, better, more powerful code than ever before.
Now, while I like to think we figured it out first, I'm not vain enough to think ideas are truly unique. So, shout out to whoever did do it first! The lesson is building data services, at the edge, on mobile is not the same as mobile development or server development; it's a hybrid.
I'll quickly drop one other cool technical feat—there are dozens more—but this post is already long, and I have several more services to cover.
Mobile apps are primarily designed to run in the foreground (open and you're looking at it). You can do a few small tasks periodically in the background, but those are for thirty-second once-a-day types of tasks. Indexing your entire digital footprint in thirty seconds, now that's a challenge! We found that we could parallelize all our HTTP requests, running upwards of two hundred transactions simultaneously. When indexing emails via Gmail, we saw a performance increase of 25x 🤯.
2. Data NFTs. So, so many data NFTs. It's no secret that companies have tried and failed to build this technology on top of existing blockchains such as Ethereum. We had to take a novel approach.
While everyone typically associates decentralized systems with blockchains like Bitcoin and Ethereum, which use consensus algorithms like Proof of Work and Proof of Stake, it's certainly not the only way. Engineers have been building decentralized systems long before the web3 revolution. Vitalik Buterin, the creator of Ethereum, wrote an excellent blog explaining decentralization. He proposes 3-types of decentralization: architectural, political, and logical. Buterin states, "Blockchains are politically decentralized (no one controls them) and architecturally decentralized (no infrastructural central point of failure) but they are logically centralized (there is one commonly agreed state and the system behaves like a single computer)."
The bottleneck, by design, is the logical centralization because architectural (and political) decentralization requires decentralized consensus. But! For our use case, logical centralization is not just unnecessary; it's a detriment. No one other than yourself needs your data ownership records until you decide to share/transact them.
User data at TIKI is decentralized in that each user's data is aggregated at the edge on their respective mobile device. By moving the ownership issuance (NFT minting) to the edge, we hyper-parallelize the process. With users minting NFTs for their data on their own devices, there is no need for decentralized consensus. In effect, each user has their own blockchain, which we appropriately call the local chain.
However, users cannot lose their data if they break/lose their devices. Users also expect the ability to use their accounts on multiple devices. The solution needs a secondary component for backup and synchronization. Yet, traditional backup and synchronization patterns create a security risk for the immutability of the NFTs. Backup tampering becomes a serious concern when a user's data is not replicated as with a traditional blockchain.
For this, we bring an old immutability pattern back from the graveyard: WORM (write-once, read many). AWS S3, Wasabi, and others use this as a service for crazy low rates like $6/mo per TB. S3's version, called Object Lock, has even been assessed for SEC Rule 17a-4(f), FINRA Rule 4511, and CFTC Regulation 1.31 by Cohasset Associates. Not too shabby. The elegance is in its simplicity; it's how you build things for mass scale.
3. Knowledge Graph. For TIKI to work, we need to make anonymous insights valuable. When Spotify first launched, one of the slogans was "better than piracy." Similar idea: our insights must be better than the old way. You only get so far with "it's more ethical" 😉.
The key is in context; it's not about who, but rather what they're doing. For perspective, global research reveals that 69% of consumers are more likely to engage with ads contextually relevant to the content they are viewing.
Instead of the traditional (non-anonymous) method of tracking identities, we pinpoint occurrences of events such as an email received, a Google search, a video watched, a package delivered, etc. You get the idea.
We then place these occurrences on our knowledge graph using both nouns and verbs as vertices.
For anyone unfamiliar, a knowledge graph is a network of entities and their relationships. The points on the graph are called vertices, and the lines connecting them are called edges. This is the underlying technology behind many search products, social networks, recommendation engines, and more. Most implementations use nouns as vertices and verbs as edges, e.g., Mike went to the Bar 🍻.
Our edges represent the number of people who experienced the event (weight). This packs in massive amounts of anonymous, searchable data without the need for huge hosting costs while simultaneously providing the contextual insights businesses covet.
I mentioned up top two additional components we're creating. The first is a system to facilitate the high-volume, anonymous, micro-transaction user payouts. We're not quite ready to unveil the plan, but you can think of it almost like a web3 clearinghouse. We're quite excited about this, and if you're deep into crypto and web3, don't hesitate to reach out for a sneak peek.
The second rolls perfectly into Part 2. We are turning all of this knowledge graph data into the largest collection of actionable business insights the world has ever seen. Imagine leveraging all this power for your business without ever needing to know anything about data — as easy as Netflix or Spotify.
It's amazing what we've been able to accomplish in such a short period of time. I'm so proud of our team, and a very serious thank you to all our investors for believing in us.
Long as we keep building on this momentum, the sky's the limit.
Mike Audi
Founder & CEO
mike@mytiki.com