CASCDR + Data Buffets Update #7: Storage Service Now Live!📦🚀🤠👍

As the hackathon draws to a close, we're still feeling inspired to add more features & capabilities! Shout out to @CoachChuckFF for leading the way on this one!

Today we are announcing the MVP of a new storage service that allows users to store arbitrary plaintext data in exchange for sats. That service can be tested on our test viewer client (source code) that can be the resource for creating any new NIP105 client!

How Does Storage Fit Into the Big Picture 📦🤔 - Part 1 Workflows Use Case:

While on the surface mundane, having the ability store data has some major implications. Fundamentally, there are three basic classes of services we could offer

(1) Data Collection: such as web scraping, downloading, survey automation etc. (a good example of this class is our Yitter Youtube downloader service)

(2) Data Processing: taking in one form of data and transforming it somehow for example using LLMs to reason/process, Whisper for voice to text transcriptions or Image2Text - taking in an image and being able to reason about that.

(3) Data Storage: while the least sexy of the three, still crucial for building stable, scalable applications.

We predict that storage will play a critical role in future developments in any decentralized computing ecosystem. The ability to pay for and make use of storage is needed in order to make apps scalable and efficient in numerous use cases. Let's illustrate with an example.

Illustrating The Power of Storage with an Example

Suppose we want to train an LLM to take on the Knowledge Base of disparate sources of information such as our company wiki, website and social media profiles. Suppose we also want that knowledge base to evolve over time and periodically retrain the model so that it is up to date with the latest information. We may wind up with a workflow something like the following:

When the workflow runs the first time it must collect the entire corpus of data which would be intensive. A necessary evil.

When it comes to updating the model based on new data however, we incur a huge efficiency penalty if we lack a storage mechanism. Without it, we are forced to recollect all the same data just like the first time. Instead, with storage we lower the burden and only bother ourselves with pulling the latest data and benefit from efficiency gains proportional to the stock of old data versus the flow of new data:

Clearly, we can see this has the potential to conserve a vast amount of resources (and sats) while still achieving the original goals we set out to. While not as exciting as other services, the storage service is clearly an essential ingredient in workflows like these.

How Does Storage Fit Into the Big Picture? Part 2 - The Future of Nostr Relay Management:

As of right now relay management while challenging has been more straightforward due to the fact that the nostr user base is relatively small. As it hopefully continues to grow, there will need to be a solution that addresses the massive increase in the collective user data footprint. Lightweight Storage as a Service APIs exchanged over sats could play a critical role in helping nostr meet the challenges that come with running a relay at scale.