Same Job, Two Engines: InnoDB and Iceberg Internals
This technical article compares the internal mechanisms of InnoDB and Iceberg, focusing on how they handle data storage, indexing, and concurrency. It explains the trade-offs between OLTP and OLAP systems, specifically highlighting how MVCC and undo logs manage concurrent read/write operations.
Why it matters
Understanding these database internals is critical for engineers optimizing data pipelines and choosing the right storage engine for specific analytical workloads.
The first version of my stock analysis repo ran analytics on an OLTP engine and my second version runs the same thing on an OLAP stack. I chose the OLAP stack because of a single optimization, but had to understand everything about the lakehouse to take advantage of it.
Rows store all columns together Fetching one bar for one symbol touches a single spot on disk, which is exactly what ingest wants. An indicator that needs two columns still fetches every column through the buffer pool.
Get smarter about the news
Sign up free for a feed built around what you actually care about, Dive Deeper research on any story, and the full text of every article.
Create free accountAlready have an account? Sign in