data-storage

MIT Technology Review examines the overlooked data storage costs of AI inference, noting that a single 8-second AI-generated video can leave behind 500MB of back-end data (130x the user-visible file) and citing IDC/McKinsey forecasts of 35% CAGR inference demand growth through 2030.

Adaptive Lossless floating-Point (ALP) Encoding is a new lightweight encoding for floating-point data in Apache Parquet, offering compression ratios similar to zstd with faster decompression, random-access support, and GPU/SIMD-friendly decoding.

This 1996 paper introduces Lifestreams, a storage model for organizing personal digital data into a continuous, searchable stream.

The tweet expresses excitement about Engram tables in DSV4.1, emphasizing their role in compression and caching techniques.

The article explains why knowledge graphs are superior to flat lists for memory storage in AI systems, as they effectively handle multiple references to the same entity and prevent fragmented or contradictory search results.

Nine PBS is facing potential loss of 50TB of archival data due to a dispute with cloud storage provider OSS, with Iron Mountain refusing access over contractual obligations.

Rivault is a product that allows AI agents to safely provide and store data and context.

Clement Delangue highlights the massive amount of data AI builders are storing on Hugging Face's Xet storage system, predicting it will reach exabytes soon. A reply frames this as a 'Stargate for Data' with labs on track to spend over $100B/year on data by 2030.

Installing the Cursor iOS app irreversibly changes your privacy settings to a less restrictive mode, with no way to revert, raising serious privacy concerns.

A proof-of-concept demonstrating how to store a small website's HTML content inside a favicon by encoding bytes into pixel RGB channels, then decoding it via JavaScript.

UTFS is a small embedded file system inspired by the TAR format, designed for microcontrollers to store data with string-based file names, separating data storage from application logic.

The article describes a creative project that stores arbitrary data (e.g., a to-do list) using iNaturalist observation IDs, encoding information through sequence and value bits in an unordered set of integers, and introduces the demo app YouDidIt.Bio.

A developer at Shopify built Quick, a zero-config API for saving data, file storage, AI, websockets, etc., running on a single VM for $200/month, changing how they work.

This article discusses and debunks a common myth about compression in Pokemon games.

Bitfield is a database claiming to be the fastest in the world with read speeds of 0.69ns and write speeds of 0.58ns.