braindump: NOSQL debrief
http://blog.oskarsson.nu/2009/06/nosql-debrief.html

NOSQL debrief

braindump: NOSQL debrief

First ever meeting of the NoSQL community. Lists all the presentations that were given.

No to SQL? Anti-database movement gains steam

The meet-up in San Francisco last month had a whiff of revolution about it, like a latter-day techie version of the American Patriots planning the Boston Tea Party. The inaugural get-together of the burgeoning NoSQL community crammed 150 attendees into a meeting room at CBS Interactive. Like the Patriots, who rebelled against Britain's heavy taxes, NoSQLers came to share how they had overthrown the tyranny of slow, expensive relational databases in favor of more efficient and cheaper ways of managing data. "Relational databases give you too much. They force you to twist your object data to fit a RDBMS [relational database management system]," said Jon Travis, principal engineer at Java toolmaker SpringSource, one of the 10 presenters at the NoSQL confab (PDF). NoSQL-based alternatives "just give you what you need," Travis said. Open source rises up The movement's chief champions are Web and Java developers, many of whom learned to get by at their cash-strapped startups without Ora

The meet-up in San Francisco last month had a whiff of revolution about it, like a latter-day techie version of the American Patriots planning the Boston Tea Party.

piece on an alternative approach to data management

Cassandra is a hybrid non-relational database in the same class as Google's BigTable. It is more featureful than a key/value store like Dynomite, but supports fewer query types than a document store like MongoDB. Cassandra was started by Facebook and later transferred to the open-source community. It is an ideal runtime database for web-scale domains like social networks.

SQL Databases Are An Overapplied Solution (And What To Use Instead)

Over the past few years, relational databases have fallen out of favor for a number of influential people in our industry. I'd like to weigh in on that, but before doing so, I'd like to give my executive summary of the events leading up to this movement

Tokyo Cabinet

Обзор нескольких опенсурсных нереляционных БД.

Thoughts on NoSQL, Tokyo Cabinet, CouchDB, Redis, and Cassandra.

Intéressant, une étude des différentes db alternatives sous l'angle de la scalabilité

http://aws.amazon.com/s3/

data store scaling technologies

Riak combines a decentralized key-value store, a flexible map/reduce engine, and a friendly HTTP/JSON query interface to provide a database ideally suited for Web applications.

answer is 3TB database???

"The fundamental problem is endemic to the relational database mindset, which places the burden of computation on reads rather than writes."

Wow, cassandra uses a lot of disk space. Trade offs!

As a Web developer it's always a good idea to know what the current practices are in the industry even if they seem a bit too crazy to adopt…yet.

bit on why non-SQL dbs are used in social networking sites

Nice detailed examples on NoSQL data modeling in Cassandra.

"Relational database management systems (DBMSs) have been remarkably successful in capturing the DBMS marketplace. To a first approximation they are “the only game in town,” and the major vendors (IBM, Oracle, and Microsoft) enjoy an overwhelming market share. They are selling “one size fits all”; i.e., a single relational engine appropriate for all DBMS needs. Moreover, the code line from all of the major vendors is quite elderly, in all cases dating from the 1980s. Hence, the major vendors sell software that is a quarter century old, and has been extended and morphed to meet today’s needs. In my opinion, these legacy systems are at the end of their useful life. They deserve to be sent to the “home for tired software.” Here’s why."

From @jesserobbins

Non-SQL oriented distributed databases are all the rage in some circles. They’re designed to scale from day 1 and offer reliability in the face of failures.

NoSQL: Distributed and Scalable Non-Relational Database Systems

l

"Most of today’s up and coming key-value stores are more than just simple key-value stores. You saw this when we looked at Tokyo Cabinet which, in addition to simple key-value capabilities, adds more sophisticated abilities, such as database-like tables. In this post we’ll look at Cassandra — a modern key-value store that continues this trend. Cassandra was originally developed by Facebook and released to open source last year. The Facebook team describes Cassandra as (Google) BigTable running on top of an Amazon Dynamo-like infrastructure."

Most of today's and up and coming key-value stores are more than just simple key-value stores. Cassandra is a modern key-value store that continues this trend.

Like mongodb but lives in memory with replication and periodic store-to-disk. Like memcached but with data structures. Great for non-critical data or replicated critical data.

Really interesting article about mongoDB and about the installation procedure

"MongoDB can be thought of as the goodness that erupts when a traditional key-value store collides with a relational database management system, mixing their essences into something that’s not quite either, but rather something novel and fascinating. -- MongoDB support is available in many languages, making it a good choice for a system that has to work in a polyglot environment; all of the major languages have support."

Good introduction to the "NoSQL" space (initially not a fan of the term, but I guess it is going to stick...), highlighting the different designs used by the options in the space, and the benefits/drawbacks of those designs.

Unprecedented data volumes are driving businesses to look at alternatives to the traditional relational database technology that has served us well for over thirty years. Collectively, these alternatives have become known as “NoSQL databases.”

"Sharding kills most of the value of a relational database."

sql database db

A nice overview of some of the more popular patterns in NoSQL architecture

Starting from Dynamo, ending with (roughly) follow @nosqlupdate on Twitter.

Materials that you need to read in order to get started with NoSQL

List of resources to read to get up-to-speed on the NoSQL movement.

Below are 7 Mongo and MongoMapper related features that I have found to be really awesome while working on switching Harmony, a new website management system by my company, Ordered List, to Mongo from MySQL.

The more I work with Mongo the more I am coming around to this way of thinking. I tell no lie when I say that I now approach Mongo with the same kind of excitement I first felt using Rails. For some, that may be enough, but for others, you probably require more than a feeling to check out a new technology

At Directi, we are taking a hard look at the way our applications need to store and retrieve data, and whether we really need to use a traditional RDBMS for all scenarios. This does not mean that we will eschew relational systems altogether. What it means is that we will use the best tool for the job – we will use non-relational options wherever needed and not throw everything at a relational database with a mindless one-size-fits-all approach. ... ... This post covers the current landscape of the NoSQL space. In a subsequent post, I intend to cover in more detail the various problem areas addressed by NoSQL systems and the specific algorithms used.

Really detailed description of a number of NoSQL solutions. Interesting reading on Cassandra and Voldemort.

This post covers the current landscape of the NoSQL space. In a subsequent post, I intend to cover in more detail the various problem areas addressed by NoSQL systems and the specific algorithms used.

FleetDB is a schema-free database optimized for agile development.

I've been a big proponent of NoSQL for a while. I have played with just about all of the new generation of data stores. We almost got cassandra running in production once, and we've been running mongodb in production for about six months now. But, here's the thing: as awesome as these new dbs are, they're still young. Our app generates a ton of data and gets pretty serious traffic. So, we started hitting walls quickly. To make a long story short, we decided to fall back to MySQL. It's battle hardened. We know its production characteristics and limitations. Backups are a science. We know we can count on it. But, we have a lot of data, and adding fields and indexes was starting to get painful. Flexible schemas are one of the things that attracted me to NoSQL in the first place. Then, I remembered this article about How FriendFeed uses MySQL to store schema-less data. So, I decided to implement the system they describe in the article. Since we put Friendly in to production, we've seen

Friendly makes MySQL look like a document store. When you save an object, it seralizes all of its attributes to JSON and stores them in a single field. To query your data, Friendly creates and maintains indexes in separate tables. It even has write-through and read-through caching built right in.

Introducing Friendly: NoSQL With MySQL in Ruby Dec 16 2009 I've been a big proponent of NoSQL for a while. I have played with just about all of the new generation of data stores. We almost got cassandra running in production once, and we've been running mongodb in production for about six months now.

25+ Alternative & Open Source Database Engines

RT @tweetlicius: 25+ Alternative & Open Source Database Engines - http://bit.ly/cRDaOW

Free Web Resources Everyday - WebResourcesDepot

Almaz

Redis' particular way of treating data requires some rethinking how to store your data to benefit from speed, atomicity and its data types. I've already written about Redis in abundance, this post's purpose is to compliment them with real-world scenarios. Maybe you can gather some ideas on how to deal with things.

Weil Redis praktisch ist.

Mongo DB Production

Interesting blog post detailing production experiences with mongodb.

RT @kvz: Why Twitter is dropping MySQL in favor of Cassandra: http://bit.ly/dyeiXF

RT @DZone "Cassandra @ Twitter: An Interview with Ryan King « MyNoSQL" http://dzone.com/WbTY

MyNoSQL: Please include anything I’ve missed.

SQL is Scalable and NoSQL Isn’t For Everyone The point is one that I think all rational people already realize: The ACID RDBMS isn’t appropriate for every need, nor is the NoSQL solution.

"[Though as Michael Stonebraker points out, SQL the query language actually has remarkably little to actually to do with the debate. It would be more clearly called NoACID]"

HyperGraphDB is a general purpose, extensible, portable, distributed, embeddable, open-source data storage mechanism. It is a graph database designed specifically for artificial intelligence and semantic web projects, it can also be used as an embedded object-oriented database for projects of all sizes.

Good discussion in the comments as well.

une base données massivement parallèle et avec l'esprit "bigtable", provient de facebook

The Apache Cassandra Project develops a highly scalable second-generation distributed database, bringing together Dynamo's fully distributed design and Bigtable's ColumnFamily-based data model.

# # Scaling practices turn a relational database into a non-relational database. To scale at Digg they followed a set of practices very similar to those used at eBay. No joins, no foreign key constraints (to scale writes), primary key look-ups only, limited range queries, and joins were done in memory. When implementing the comment feature a 4,000 percent increase in performance was created by sorting in PHP instead of MySQL. All this effort required to make a relational database scale basically meant you were using a non-relational database anyway. So why not just use a non-relational database from the start?

As Digg started out with a MySQL oriented architecture and has recently been moving full speed to Cassandra, his observations on some of their lessons learned and the motivation for the move are especially valuable. Here are some of the key takeaways you find useful:

RT @Sebdz: RT: @programmateur: Digg: 4000 % performance increase by sorting in PHP rather than MySQL (via @mrboo) - http://bit.ly/ckma10

♻ @n1k0: "Scaling practices turn a relational database into a non-relational database" http://n1k.li/4v (via @nsilberman)

Typically for relatively static data sets, relatively low query volumes, and relatively high latency requirements.

Introductory blog post about the Cassandra data model.

posted by thraxil: http://quimby.ccnmtl.columbia.edu/ircbot/web/?y=2010&m=04&d=26#20100426105402

Awesome tutorial.

These slides and notes were originally written to accompany a three hour Redis tutorial I gave at the NoSQL Europe conference on the 22nd of April 2010.

HBase vs Cassandra: why we moved

Maybe I should learn to use Cassandra someday.

SCALABLE, OPEN-SOURCE SQL DBMS WITH ACID

A discussion of the different NoSQL-style datastores in use at Twitter, including Hadoop (with Pig for analysis), HBase,

Twitters NoSQL slides

A discussion of the different NoSQL-style datastores in use at Twitter, including Hadoop (with Pig for analysis), HBase, Cassandra, and FlockDB.

cassandra,thrift, hdfs, hbase, scribe,pig,lzo, flockdb

interesting presentation on #NoSQL at #twitter by @kevinweil http://bit.ly/99h8BK [from http://twitter.com/behi_at/statuses/13587582774]

For those familiar with memcached, membase provides on-the-wire protocol compatibility, but adds disk persistence; hierarchical storage management; data replication; live cluster reconfiguration and rebalancing; and secure multi-tenancy with data partitioning. Like memcached, membase is simple, fast and elastic.

Persistent Key/Value Storage

Membase is an open-source (Apache 2.0 license) distributed, key-value database management system optimized for storing data behind interactive web applications. These applications must service many concurrent users; creating, storing, retrieving, aggregating, manipulating and presenting data in real-time. Supporting these requirements, membase processes data operations with quasi-deterministic low latency and high sustained throughput.

from oreilly news link

Membase is an open-source (Apache 2.0 license) distributed, key-value database management system optimized for storing data behind interactive web applications. These applications must service many concurrent users; creating, storing, retrieving, aggregating, manipulating and presenting data in real-time. Supporting these requirements, membase processes data operations with quasi-deterministic low latency and high sustained throughput. It scales linearly from a single-server deployment to a cluster of thousands of machines. And because membase does not require creation of a schema before storing data, it is a flexible, cost-effective place to Store Lots of Stuff.

Membase is an open-source (Apache 2.0 license) distributed, key-value database management system optimized for storing data behind interactive web applications. These applications must service many concurrent users; creating, storing, retrieving, aggregating, manipulating and presenting data in real-time. Supporting these requirements, membase processes data operations with quasi-deterministic low latency and high sustained throughput. It scales linearly from a single-server deployment to a cluster of thousands of machines. And because membase does not require creation of a schema before storing data, it is a flexible, cost-effective place to Store Lots of Stuff. The original membase source code was released as Open Source by NorthScale, Zynga and NHN to membase.org in June 2010.

PlayNice.ly is entirely based on a data-structure server called Redis. Redis is one of several new key-value databases which break away from traditional relational data architecture. It is simple, flexible, and blazingly fast. So why not use the tools we have already?

redis.smembers("word:" + metaphone("python"))

Interesting post about being able to search data in redis using indexing and phonetic algorthms.

Reflections on MongoDB -- http://bit.ly/aHCUC9