TIKI SDK Tech Updates
Are you following the TIKI Discord? If not, you are missing weekly tech updates from Ricardo and weekly CEO updates from Mike. Never fear, if you don’t feel like joining our server and downloading another application, I’ve gone ahead and compiled the recent tech updates from Discord and migrated them here to Substack. I’ll do this at appropriate times in the future for those who want to catch up in bulk. If you want the weekly updates, join the Discord or check out our Twitter account. I will soon be posting abbreviated versions of Ricardo’s updates as threads.
TIKI is currently finalizing the productization of our technology in the form of an SDK to enable other companies to add our decentralized data exchange (DEX) to their applications. Our DEX opens up new data features such as loyalty programs and data DAOs, increasing opt-in rates for things like IDFA and mitigating data risk with on-chain consent.
We are beyond excited about this product and believe it to be an intuitive, easy-to-use, and most importantly, desirable entry point for anyone to join the data ownership movement.
Take a read through our mobile lead Ricardo’s journey of developing and (soon) launching the TIKI SDK. Transparency, one of the three pillars of TIKI, allows for a unique opportunity to get inside the minds of our team members as we work toward “unfucking the internet.”
- Shane
August 12th, 2022
Last week Mike presented our blockchain white paper. So, what's the plan now? Let's build it!
This week I delved into this document and started writing our first SDK. It will allow the use of blockchain in various scenarios to give our users ownership of their data, control over it, and compensation for its use.
We decided to start building the SDK from the ground up, which went right through the blockchain implementation. It will follow the slice architecture we already use in all other packages. Each functionality is self-contained making it independent of the rest of the code. This allows each piece of code to be super specialized and not need to communicate with other pieces to do what it was programmed to do.
The implementation is divided into a few slices:
- transactions - which are the most basic records of the blockchain
- blocks - which actually form the blockchain (that's where the “block” comes from hahaha)
- node that takes care of blockchain logic
- crypto which is responsible for all operations involving cryptography
- keystore - that keeps the users' private key with the most suitable implementation according to the operating system where it runs.
- backup - responsible for making the remote backup of the chain
- xchain - which will command the synchronization between local and remote chains.
You can follow the development of the SDK through the repository https://github.com/tiki/tiki-sdk-dart/
August 18th, 2022
Updates in the blockchain development!
We have the data structure defined and implemented for the blocks, transactions, cross chain references and backups. 🥳
Yesterday we did a detailed code review and finalized the underlying data structure for blocks, transactions, backups and cross-chain references.
Mike is working on layer0 of the blockchain, coordinating authorization, data backup and communication with object storage, while I am building the logic for handling the transactions and blocks in our chain.
The goal is to finalize our SDK's blockchain implementation soon, so we can move on to the next layer, which is the creation of data ownership NFTs and data usage consent NFTs 🤤.
We still have a lot of work to do! I’ll keep you posted!
August 24th, 2022
In the last 2 weeks I have added a new skill in my LinkedIn profile: Blockchain engineer!
We are in the final sprint to have our first blockchain implementation, written in pure Dart, that will be used for mobile, server and web SDKs to build data ownership and consent NFTs.
All the low level code for cryptography, block and transactions operations are in place. It already creates a transaction, signs it, build the proofs of ownership and integrity and adds it to a block that become part of the chain.
We are now working in the final APIs for the node management and communication with remote object storage that will be our default backup for all the chains.
I am really excited with all that we are building! It will be available soon for anyone that wants to implement data consent and ownership in their own software through TIKI.
September 1st, 2022
After a few weeks of learning, testing, and improving, we are finally approaching version 1.0 of our blockchain!
And with this event approaching, I would like to share a little bit of the technical part, which (almost) nobody talks about in the crypto world: how does the blockchain code work?
Yes, we know that the blockchain is a chain of blocks where the next block always refers to the previous one, which prevents any block in the chain from being changed without the whole data structure change. But nobody talks about all the processes that involve writing unalterable data in a chain of blocks.
Following our blockchain specification, each block is made up of a list of transactions that are the containers of the data. To create a new record, we create a transaction using the data. The SDK uses the user's private key to sign this transaction creating a string that can only be recreated using my private key and can be verified using my public key. Once the transaction is signed, we create another string that is the unique identification of the transaction, better known as a “hash”. This sequence is created using the transaction bytes and the SHA3 algorithm, which makes it mathematically unlikely that two different transactions have the same hash. The transaction is then placed in a list so that it can be added to the next block to be created.
Blocks are created one minute after the oldest transaction is placed on the list or when the transaction list is larger than 100KB. To create a block, the SDK uses the transaction list to calculate the reduction of all transaction hashes into one. For this, we use a data structure called Merkel Tree that reduces the hashes in pairs until only one is left. That last hash is called the Merkel Root and is saved as the root of transactions that block.
When calculating the root of transactions, we also calculate the Merkel Proof of each transaction, that is, the minimum sequence of calculations that needs to be done to arrive at the Merkel Root, from each transaction. This value is used to make checking the transaction's inclusion in the block faster than having to recalculate the entire Merkel Tree. The Merkel Proof of the transaction is not included in the blockchain but is saved in the local cache.
Once we have the transaction root we can create our block. For this, we use the id of the previous block to create the chain link. With the transaction root and the hash of the previous block, we can create the hash of this block, which will be its unique id on the blockchain. And so we created our block!
For now, these blocks and transactions are still only on the device that created them. As we use the concept of multiple chains, each device has its blockchain. To save them in a remote backup and make access to this data public, we create a new chain record using the public key address and save the public key itself in an object storage. With this record created, we can start adding blocks and transactions to that same store, to make them available to anyone on tiki's internal network.
And that's what we're working on now. Sending this data to object storage and synchronizing multiple chains on the user's device.
We are getting close, I will bring news soon…
September 8th, 2022
Delayed tech update! And today's subject is precisely why the tech update is delayed.
Most of my professional experience is as a high-level programmer. It has nothing to do with my abilities. High-level programmers are the ones who work with data structures further away from pure bit-and-byte computation. High-level programmers work with languages and features closer to the functional requirements of the systems, i.e., features closer to the use cases of the software. In other words, more focused on what it does. Low-level programmers work closer to nonfunctional system requirements and the computation itself, that is, features related to how the software works.
But a programmer is a programmer and no matter the level, let's turn code into a product!
With the implementation of blockchain, I have gotten much closer to the low level of programming, working with cryptography, bits, and bytes. Mike has already done quite a lot of the hard work for the blockchain on our current app. We are reusing some of this code in the SDK but I am responsible now for enhancing it. How challenging it has been! And by challenging I mean exciting, fascinating, enriching, and... tiring!
Working with blockchain we have an unknown amount of data that will be stored. It could be a few transactions or millions of them. When you store a large amount of data you need to be careful to use the smallest way to store that data. And that is to store the smallest amount of bytes needed to represent it. Then we convert the data that is structured in the system into a sequence of bytes and put one sequence after another. This is how we store transactions and blocks.
But when we are going to retrieve this data, how do we know which byte starts or ends each transaction or block? Simple, the first byte will store the transaction or block size in bytes. So if the first byte has a value of 100, we know that from byte 2 to 101 we have the first transaction and that in byte 102 we will have the size of the second transaction, and so on. Simple no? Not so much.
It turns out that a byte is a sequence of 8 bits that can represent the maximum number 255 (2ˆ8). But what if the size of the sequence of bytes that the transaction occupies is greater than 255, how do we know? This is where Compact Size comes in.
With Compact Size, we use the first byte to inform the size of the number that will identify the size of the transaction.
If the first byte is less than or equal to 252, it means that the number representing the size of the next data is less than or equal to 252. Since 252 fits in one byte, the first byte is exactly the size of the data.
If it is 253, it means that the number is greater than 252 and less than or equal to 65535. This means that it fits in 2 bytes (2^16). Then we use the next 2 bytes to store the number.
If it is 254, the number is between 65535 and 4,294,967,295 and needs the next 4 bytes (2^32). And if the first byte is 255, the transaction size is greater than 4,294,967,295 and less than 18,446,744,073,709,551,615 and needs to be represented in 8 bytes (2^64).
Simple explanation, right? And it will be easy to implement. I thought. It was not. Transforming integers to 8-bit to 64-bit byte lists in a language that only works with 32-bit or 64-bit integers, depending on the OS, took a lot longer than the 60 minutes I had allotted. It took a whole day!! And it wasn't until the end of the day that I realized I was doing it all wrong...
Anyway, it was a great learning experience and this is part of what we do here. But at the end of the day, I was exhausted and didn't have the time to write the tech update. And that's why it was just posted today.
PS.: I haven't forgotten about the bugs you guys have been sending in #bug-reports. I'm putting them all on the list so that as soon as I have a little time in the development of the SDK, I can fix them all. And soon we will have a brand new app for you!
September 21st, 2022
Tech update: We have a blockchain node, and docs!
I promise that I will not get (too much) into technical stuff today... : )
Last week we have finished the 0.0.3 version of the Dart SDK which includes all the basic blockchain operations for the node. In this version the SDK is capable to write and read transactions, blocks and verify the chain integrity and ownership.
With so much to do, we sometimes miss the importance of the milestones that are achieved. And this is a big one. This is the first version of the blockchain node in the SDK that will be used by everything else that is coming after it. We know that we have a lot of room for improvement, and we are working already in new features and improvements for it.
And with version 0.0.3, we now have docs! It is the technical API that describes how SDK code works. It is a technical documentation, and is very important for other developers to understand how to use de SDK and for transparency.
If you want to read some of our code and understand how it works, please take a look. It will be updated with every new release. https://tiki.github.io/tiki-sdk-dart/
September 28th, 2022
And here it come Ownership and Consent NFTs!!
It is not big news that we have data ownership NFTs in TIKI. We have been minting millions of those for almost 5 months now in our app. The news is that now we have Ownership and Consent NFTs in the SDK built on top of our blockchain implementation. Which means that ANY business in the world will be able to (and SHOULD) use TIKI SDK to create data NFTs in the user's behalf.
And how does it work?
First, it registers an NFT that states the user as the owner of a specific data asset created by the business. It could be a single data point, like a purchase, a data stream, like all the purchases one does from now on, or a data pool, like all the purchase history. It is the Ownership NFT.
After that, the business needs to ask for your consent to use that data. In exchange, it can reward you with discounts, loyalty points or whatever it wants. With your consent, the SDK creates a new NFT that registers you have consented that the business can use that specific data in specific use cases in exchange for the reward they have proposed. That is the Consent NFT.
And in any given point, you can change your mind about any consent given to any business in the world. You need to modify given consent by creating a new Consent NFT.
We are working in this piece of code as I write this. It will get into our SDK in the next few days.
October 5th, 2022
Now that we are done with data Ownership and Consent NFTs creation, we need to make it easy to implement TIKI SDK in other software.
This week we are working on the APIs for TIKI SDK for Android and iOS. Since our base SDK is written in Dart, the first step is to build a Flutter implementation that can expose it to be compiled into Android and iOS apps.
But we want that every Android or iOS project in the world can use our SDK not just the ones written in Flutter. That's why right after the Flutter implementation is done, we will create platform-specific channels for Android and iOS. This structure makes it possible for Android native code, written in Kotlin or Java, or iOS native code, written in Swift or Objective-C, can call our SDK without having to use the Flutter SDK.
Stay tuned! We are getting there...
October 13th, 2022
This is the final coding week for TIKI SDK 1.0! I am so focused in the code and excited by seeing the SDK approaching its mature state, that I forgot to post the update yesterday, but here it is!
The Flutter SDK is ready, iOS and Android native code channels are working in the Flutter SDK. Any Flutter developer can already depend on the TIKI SDK Flutter 0.0.6 to create the data NFTs for ownership and consent! Now we are wrapping this code into native libraries for Android and iOS.
For Android, our SDK will be delivered as a Android AAR module. It is a kind of Java library (jar), wrapped with Android-specific code. It is going to be published in the Maven Central repository, a well-known repository for public shared libraries, and any Android developer will be able to add TIKI SDK to their app.
For iOS, the SDK is a Cocoa Pod package. Although we love chocolate at TIKI, this naming is not something specific to us. Cocoa is one of the application environment for OSx and the only one for iOS. The Cocoa Pod is the public dependency manager for Swift and Objective-C Cocoa projects. With TIKI SDK published there, it will be possible for any iOS developer to use it.
The next steps towards the TIKI SDK 1.0 lunch are more about technical documentation and code samples. And fixing bugs, of course... I will keep you posted! 1.0 is coming!!!
EDIT: actually the correct TIKI SDK Flutter version is 0.0.3... it is ready for use in Flutter but still in active development for native code. The 0.0.6 version is the Dart SDK, which is in stable version already, waiting for the other libs to be pushed to 1.0 version.