Redis Cookbook: Practical Techniques for Fast Data Manipulation

by Tiago Macedo, Fred Oliveira

Two years since its initial release, Redis already has an impressive list of adopters, including Engine Yard, GitHub, Craigslist, and Digg. This open source data structure server is built for speed and flexibility, making it ideal for many applications. If you're using Redis, or considering it, this concise cookbook provides recipes for a variety of issues you're likely to face.

Each recipe solves a specific problem, and provides an in-depth discussion of how the solution works. You’ll discover that Redis, while simple in nature, offers extensive functionality for manipulating and storing data.


Learn when it makes sense to use Redis
Explore several methods for installing Redis
Connect to Redis in a number of ways, ranging from the command line to popular languages such as Python and Ruby
Solve a range of needs, from linked datasets to analytics
Handle backups, sharding, datasets larger than available memory, and many other tasks

Genres: Programming, Technology, Software

74 pages, Paperback

First published January 1, 2011

Reviews

[Gafitescu Daniel · Rating 4 out of 5]

Installing Redis


1. Downloading the source
You can download Redis from the official site or directly from the Github
project, either using Git or your browser to fetch a snapshot of one the branches
or tags. This allows you get to get development versions, release candidates, etc.
2. Compiling
Redis compilation is straightforward. The only required tools should be a C com-
piler (normally GCC) and Make. If you want to run the test suite, you also need
Tcl 8.5.
After unpacking the source and changing your terminal path to the source direc-
tory, just type:
make
This will compile Redis, which on a modern computer should take less than 10
seconds. If you’re using a x86_64 system but would like an x86 build (which uses
less memory but also has a much lower memory limit), you can do so by passing
along 32bit to Make:
make 32bit
After compiling Redis, particularly if you’re building a development version, you
should run the test suite to ensure that the server is behaving as expected.
make test
3. Installing
After compiling Redis, you can go ahead and run it:
cd src && ./redis-server
However, you might find it more convenient to install it to another location in your
system. The Makefile wlll also help you do that:
make install

This will install Redis binaries to /usr/local/bin. If you wish to install to another
location, you can pass it to make. For instance:
make install /opt/local
This will install the binaries in /opt/local/bin.
After installating the Redis server, you should also copy the configuration file
(redis.conf) to a path of your choice, the default being /etc/redis.conf. If your con-
figuration file is in a different path from the default, you can pass it along as a
parameter to redis-server:
/usr/local/bin/redis-server /alternate-location-for-redis-config.conf


Installing on Linux
Debian/Ubuntu
sudo apt-get install redis-server
Fedora/Redhat/CentOS
sudo yum install redis
Gentoo
sudo emerge redis

Using Redis Data Types


Redis includes
several built-in data types, allowing developers to structure their data in meaningful
semantic ways. Predefined data types add the benefit of being able to perform data-
type specific operations inside Redis, which is typically faster than processing the data
externally

• Be consistent when defining your key space. Because a key can contain any char-
acters, you can use separators to define a namespace with a semantic value for your
business. An example might be using cache:project:319:tasks, where the colon
acts as a namespace separator.

• When defining your keys, try to limit them to a reasonable size. Retrieving a key
from storage requires comparison operations, so keeping keys as small as possible
is a good idea. Additionally, smaller keys are more effective in terms of memory
usage.

• Even though keys shouldn’t be exceptionally large, there are no big performance
improvements for extremely small keys. This means you should design your keys
in such a way that combines readability (to help you) and regular key sizes (to help
Redis).

Strings
The simplest data type in Redis is a string. Strings are also the typical (and frequently
the sole) data type in other key-value storage engines. You can store strings of any kind,
including binary data. You might, for example, want to cache image data for avatars
in a social network. The only thing you need to keep in mind is that a specific value
inside Redis shouldn’t go beyond 512MB of data.

Lists
Lists in Redis are ordered lists of binary safe strings, implemented on the idea of a linked
list. This means that while getting an element by a specific index is a slow operation,
adding to the head or tail of the data structure is extremely fast, as it should be in a
database. You might want to use lists in order to implement structures such as queues,
a recipe for which we’ll look into later in the book.


Hashes
Much like traditional hashtables, hashes in Redis store several fields and their values
inside a specific key. Hashes are a perfect option to map complex objects inside Redis,
by using fields for object attributes (example fields for a car object might be “color”,
“brand”, “license plate”).

Sets and Sorted Sets
Sets in Redis are an unordered collection of binary-safe strings. Elements in a given set
can have no duplicates. For instance, if you try to add an element wheel to a set twice,
Redis will ignore the second operation. Sets allow you to perform typical set operations
such as intersections and unions.
While these might look similar to lists, their implementation is quite different and they
are suited to different needs due to the different operations they make available. Mem-
ory usage should be higher than when using lists.
Sorted sets are a particular case of the set implementation that are defined by a score
in addition to the typical binary-safe string. This score allows you to retrieve an ordered
list of elements by using the ZRANGE command. We’ll look at some example applications
for both sets and sorted sets later in this book.

Using Redis from the Command Line

redis-cli -h


The most typical usage scenarios would be something like the following, to connect to
a remote server in interactive mode:

redis-cli -h serverip

The following connects to a local server running on a nondefault port in interactive
mode:
redis-cli -p 6380

The following connects to a local server on the default port (6379), executes the
INFO command, and returns you to your original shell:
redis-cli INFO


Using Redis as a Key/Value Store

- Storing application usage counters ( storing something quite basic: counters. Imagine we run a business social network and want to track profile/page visit data. )


- we can use commands such as INCR (or INCRBY) and DECR (or DECRBY) to
increment or decrement its contained value.

To store our social network page visit data, we could have a key
namespace such as visits:pageid:totals, which for a page ID of 635 would look like
visits:635:totals.

If for instance we need to switch from the mysql set the current values in REDIS :

SET visits:1:totals 21389

On a visit to a given page, a simple INCR command would update the counter in Redis:
INCR visits:635:totals

We could then grab the page visits for any page, at any time by doing a simple GET
command by key:

GET visits:635:totals

you can take advantage of the return value from the INCR command because it returns the
post-increment count. A simple pseudocode for visits and counters could look like this:
1. The visitor requests the page.
2. We INCR the visits counter related to the page (INCR visits:635:totals, for in-
stance).
3. We capture the return value of the INCR command.
4. We show the user the page with the return value

Ex :

$redis = new Predis_Client($single_server);
$nr_visits = $redis->incr('nr_visits');

Storing object data in hashes

We’ll begin by designing a key namespace to store our users. As before, we’ll be sepa-
rating keywords with colons to generate a rich key that makes sense in our system. For
the sake of this recipe, we’ll go with something simple like keys in the form of
users:alias, where alias is a binary-safe string. So to store information about a user
called John Doe, we might build a hash called users:jdoe.
Let’s also assume we want to store a number of fields about our users, such as a full
name, email address, phone number, and number of visits to our application. We’ll
use Redis’s hash management commands—like HSET, HGET, and HINCRBY—to store this
information.

redis> hset users:jdoe name "John Doe"
(integer) 1
redis> hset users:jdoe email "jdoe@test.com"
(integer) 1
redis> hset users:jdoe phone "+1555313940"
(integer) 1
redis> hincrby users:jdoe visits 1
(integer) 1
With our hash built and in place, we can fetch single fields with HGET or the full hash
by using the HGETALL command, as exemplified here:

redis> hget users:jdoe email
"jdoe@test.com"
redis> hgetall users:jdoe
1) "name"
2) "John Doe"
3) "email"
4) "jdoe@test.com"
5) "phone"
6) "+1555313940"
7) "visits"
8) "1"

There are auxiliary commands like HKEYS, which return the keys stored in a particular
hash, and HVALS, which returns only the values. Depending on how you want to retrieve
your data, you may find it useful to use HGETALL or one of these to retrieve data from
Redis into your application.

redis> hkeys users:jdoe
1) "name"
2) "email"
3) "phone"
4) "visits"
redis> hvals users:jdoe
1) "John Doe"
2) "jdoe@test.com"
3) "+1555313940"
4) "1"

Sets

Ex:

Adding to a set. Redis’s support for sets to create functionality similar to the circles in the
recently launched Google+.

Add to sets

sadd

redis> sadd circle:jdoe:family users:anna
(integer) 1
redis> sadd circle:jdoe:family users:richard
(integer) 1
redis> sadd circle:jdoe:family users:felix
(integer) 1


$this->redis->sadd('circle:jdoe:soccer', 'users:john');
$this->redis->sadd('circle:jdoe:soccer', 'users:anna');
$this->redis->sadd('circle:jdoe:soccer', 'users:james');
$this->redis->sadd('circle:jdoe:soccer', 'users:richers');

$data = $this->redis->smembers('circle:jdoe:family');

Array
(
[0] => users:adam
[1] => users:felix
[2] => users:richard
[3] => users:anna
)

Get elements from the sets

smembers circle:jdoe:family

Intersection between sets

Command : sinter

sinter circle:jdoe:family circle:jdoe:soccer

Union between sets

Command : sunion

sunion circle:jdoe:family circle:jdoe:soccer

The Redis command that allows you to list your data is the KEYS command. Use it with
the supported wildcard matchers. Thus, the following command:
KEYS *


$data = $this->redis->keys('*');

However, that is not enough, as you still may
not know what the key type is. That’s what the TYPE command is for:
TYPE keyname
This will tell you whether that key is a string, hash, list, set, or zset.

$data = $this->redis->type('users:jdoe');
debug($data);

The wildcard syntax of the KEYS command is limited but quite useful. It supports queries
like:
KEYS h*llo
Returns all keys starting in h and ending in llo.
KEYS h?llo
Returns keys that start with h, end with llo, and have exactly one character between
them.
KEYS h[ae]llo
Returns only the keys hallo and hello, if they exist.


HSET hash-name key value
Sets a value on a hash with the given key. As with other Redis commands, if the
hash doesn’t exist, it’s created.


(integer) 1
redis> HGET myhash field1
"Hello"
redis>


$this->redis->hset('user',"username","gafitescu");
$this->redis->hset('user',"email","james@james");
$this->redis->hset('user',"address","With Google Chrome, Firefox 4 or higher, or Internet Explore");


$data = $this->redis->hget('user','address');
debug($data); => With Google Chrome, Firefox 4 or higher, or Internet Explore

$data_all = $this->redis->hgetall('user');
debug($data_all); =>
Array
(
[username] => gafitescu
[email] => james@james
[address] => With Google Chrome, Firefox 4 or higher, or Internet Explore
)

HMSET hash-name key1 value1 [key2 value2 ...]
Allows you to set several values in a hash with a single command.

redis> HMSET myhash field1 "Hello" field2 "World"
OK
redis> HGET myhash field1
"Hello"
redis> HGET myhash field2
"World"
redis>

$this->redis->hmset('user_info',array("first_name" => "James",
"last_name" => "Joyce",
"sex" => "M",
"age" => 15));
$data = $this->redis->hget('user_info','last_name');
debug($data); => string(5) "Joyce"


$data_all = $this->redis->hgetall('user_info');
debug($data_all); =>
Array
(
[first_name] => James
[last_name] => Joyce
[sex] => M
[age] => 15
)

EXPIRE key seconds
Sets an expiration timeout on a key, after which it will be deleted. This can be used
on any type of key (strings, hashes, lists, sets or sorted sets) and is one of the most
powerful Redis features.
EXPIREAT key timestamp
Performs the same operation as EXPIRE, except you can specify a UNIX timestamp
(seconds since midnight, January 1, 1970) instead of the number of elapsed sec-onds.
TTL key
Tells you the remaining time to live of a key with an expiration timeout.
PERSIST key
Removes the expiration timeout on the given key.

$data_all = $this->redis->hgetall('user');
debug($data_all);
$this->redis->expireat('user',1418434062 );

$ttl = $this->redis->ttl('user' );

debug($ttl,1);

HDEL hash-name key
Deletes a key/value pair in the given hash.

$this->redis->hmset('user_info',array("first_name" => "James",
"last_name" => "Joyce",
"sex" => "M",
"age" => 15));
$data_all = $this->redis->hgetall('user_info');
debug($data_all);

$this->redis->hdel('user_info','sex');
$data_all = $this->redis->hgetall('user_info');
debug($data_all);

=>
Array
(
[first_name] => James
[last_name] => Joyce
[age] => 15
)


You want to leverage Redis’s pub/sub functionality to create a light real-time chat sys-
tem with Node.js and Socket.IO.

ZINCRBY zset-name increment element
Adds or increments the score of an element in a sorted set. As with ZADD and SADD, the set will be created if it doesn’t exist.

$this->redis->zincrby('view_counters',1, 'Big bang theory');
$this->redis->zincrby('view_counters', 1,'Pan Am');
$this->redis->zincrby('view_counters', 1,'Seifield');
$this->redis->zincrby('view_counters', 1,'Friends');
$this->redis->zincrby('view_counters',1 ,'james');

$nr = $this->redis->zincrby('view_counters', 1,'Friends');
var_dump($nr);

HMGET hash-name field1 [field2 ...]
Fetches several fields from a given hash. This command is similar to HGET, but
allows you to get several fields in a single operation.


$this->redis->hmset('metavars', 'foo', 'bar', 'hoge', 'piyo');
$data = $this->redis->hmget('metavars', 'foo', 'hoge');
debug($data); =>
Array
(
[0] => bar
[1] => piyo
)


ZINTERSTORE destination-zset number-of-zsets-to-intersect zset1 [zset2 ...]
[WEIGHTS weight1 [weight2 ...]] [AGGREGATE SUM | MIN | MAX]
Gets the intersection of a given number of ZSETS and store the result in a new
ZSET. It’s also possible to pass along a muliplication factor for each ZSET
(WEIGHTS) or to specify the aggregation function. By default, it’s a sum of the scores
in all the sets, but it can also be the maximum or minimum value.

ZREVRANGE zset-name start-index stop-index [WITHSCORES]
Returns the elements in the sorted set within the given range, in descending order.
The command can also optionally include the scores of the elements in the returned
result.
Array
(
[0] => Array
(
[0] => Kiki ricky micky
[1] => 124
)

[1] => Array
(
[0] => Pan2 Am
[1] => 120
)

[2] => Array
(
[0] => 1st serie
[1] => 116
)

[3] => Array
(
[0] => Sei2field
[1] => 114
)

[4] => Array
(
[0] => Fr2iends
[1] => 108
)

)

$range = $this->redis->zrevrange('show_views_current',0,1, 'withscores'); => get only 2 first elements

The ZRANGE command performs the same operation, but in ascending order.

$this->redis->zincrby('show_views_current', rand(1,10),'1st serie');
$this->redis->zincrby('show_views_current', rand(1,10),'Pan2 Am');
$this->redis->zincrby('show_views_current', rand(1,10),'Sei2field');
$this->redis->zincrby('show_views_current', rand(1,10),'Fr2iends');
$this->redis->zincrby('show_views_current', rand(1,10),'Kiki ricky micky');
$range = $this->redis->zrange('show_views_current',0,-1, 'withscores');
debug($range);
Array
(
[0] => Array
(
[0] => Fr2iends
[1] => 88
)

[1] => Array
(
[0] => 1st serie
[1] => 100
)

[2] => Array
(
[0] => Sei2field
[1] => 105
)

[3] => Array
(
[0] => Pan2 Am
[1] => 108
)

[4] => Array
(
[0] => Kiki ricky micky
[1] => 115
)

)
ZRANK set-name member
Returns the rank (index) of the given member in the sorted set.

$rank = $this->redis->zrank('show_views_current', 'Sei2field');
debug($rank); ⇒ 2

EXISTS key
Checks for the existence of a key. Returns 1 i the key exists, and 0 if it does not.


$exists = $this->redis->exists( 'show_views_current'); => true
$exists = $this->redis->exists( 'show_views_current_here'); => false

Lists


A typical use case for Redis has been a queue on top of lists.



RPUSH list-name value
Inserts the given value at the tail of the list-name list. Should this list be nil, it will
be created.

$list = "lista";
$this->redis->rpush($list,"first");


LRANGE list-name start-index stop-index
Returns the list elements in the specified range (including the rightmost element
specified).

$list = $this->redis->lrange($list,0,-1 ); =>

Array
(
[0] => first
[1] => first
[2] => second
[3] => third
)


$this->redis->lrange($list,0,2 ); =>

Array
(
[0] => one element
[1] => first
[2] => first
)


LPUSH list-name value
Like RPUSH, but inserts the element at the head o f the list

$this->redis->lpush($list,"one element");
Array
(
[0] => one element
[1] => first
[2] => first
[3] => second
[4] => third
)

LLEN list-name
Returns the length of the given list.
$length = $this->redis->llen($list);
debug($length,1);

LREM list-name count value
Removes the first count occurrences of elements equal to value from the list stored at key. The count argument influences the operation in the following ways:
count > 0: Remove elements equal to value moving from head to tail.
count < 0: Remove elements equal to value moving from tail to head.
count = 0: Remove all elements equal to value.

For example, LREM list -2 "hello" will remove the last two occurrences of "hello" in the list stored at list.
Note that non-existing keys are treated like empty lists, so when key does not exist, the command will always return 0.


$arList = $this->redis->lrange($list,0,-1 );
debug($arList);

$this->redis->lrem($list,0,'first');

$arList = $this->redis->lrange($list,0,-1 );
debug($arList);



Array
(
[0] => one element
[1] => first
[2] => first
[3] => second
[4] => third
)

After removal

Array
(
[0] => one element
[1] => second
[2] => third
)

LTRIM list-name start-index stop-index
Trims the list so that it only contains the elements in the specified range. It’s similar
to the LRANGE command, but instead of just returning the elements, it trims the list.

$this->redis->ltrim($list,1,1 );

LPOP list-name
Removes and returns the element at the head of the list.
$this->redis->lpop($list);


RPOP list-name
Like LPOP, but performs the action at the tail of the list.

$this->redis->rpop($list);



Backing up the files

In case you want an up-to-date snapshot (instead of using the last one Redis did ac-
cording to your settings) you can trigger it by issuing:
redis-cli BGSAVE
File is saved here /var/lib/redis/dump.rdb

[MBybee · Rating 4 out of 5]

Nice basic intro. I liked it, and having bought it as a PDF it'll be nice to skim/search around in later.
Pretty ok as a beginning.

[Yusup · Rating 1 out of 5]

pretty basic stuff

Not recommended, 80 pages of selective content from the official docs. Buy something else or just read the official docs.

[Rober · Rating 3 out of 5]

It has some nice examples to get started. It is not the book to get a deeper knowledge of redis, but it's the next step after reading a redis primer or the original Redis in Action.

[Alessandro Pellizzari · Rating 2 out of 5]

Partiamo dai pregi: i capitoli sull'amministrazione di un server sono abbastanza interessanti.

Tutto il resto, purtroppo, sono difetti.

A partire dagli esempi, basati su concetti non proprio semplici: non puoi spiegare come salvare i dati di OAuth senza spiegare come funziona OAuth, e dandolo per scontato, o fare esempi in Ruby (invece che nel normale linguaggio della console di Redis) mettendoci anche generatori, espressioni lambda e la tipica sintassi di Ruby che nessun altro linguaggio usa e pertanto risulta criptica.

In generale, poi, le spiegazioni sono davvero striminzite, anche nei capitoli sull'amministrazione di sistema. Per esempio dice che si può fare backup tramite uno snapshot, ma non dove andare a prendere lo snapshot, come copiarlo da qualche altra parte, come fare restore pulito, ecc.

Forse era limitato a livello editoriale, forse ha voluto parlare di troppi argomenti, o forse semplicemente aveva in testa come si fanno le cose ma non ha saputo spiegarle.

Una delle poche delusioni dai libri O'Reilly.

[André Hagenbruch · Rating 3 out of 5]

Overall this is a good overview with some nice use cases. You'll profit most from the code examples if you know some Ruby but the examples are not that esoteric that you won't understand them if you don't. What I'm missing most from the book is an index so you can get from a command to the use case and/or the quick reference that accompanies some of the recipes. Another niggle I have: you would think that it's pretty easy to catch typos in a volume as slim as this one but be prepared to encounter some of them especially towards the end...;-)

[Paul Barry · Rating 4 out of 5]

A nice, short book with some great practical use cases for Redis

[Alex Ott · Rating 3 out of 5]

short book, that shows how to use Redis to solve some tasks.

P.S. (really between 3 & 4)

[Piotr Zurek · Rating 3 out of 5]

Shot book with very basic information in it. Good quick read if you want to get an idea what Redis is and what to use it for.

[Daniel González · Rating 1 out of 5]

too basic