Introduction
Redis sorted set (ZSET) is a powerful data structure that allows you to create highly responsive and scalable leaderboards. Traditionally, the ZSETs were primarily associated with gaming applications. However, you can use the Redis leaderboards for many applications in today's evolving IT industry.
For instance, you can create a scoreboard to log the total revenue generated by each salesperson and their rank compared to other staff. This enhances healthy competition in your sales department. Similarly, you can implement a leaderboard in a fitness tracking app to encourage members to complete their goals. For example, you can log their step counts or any other exercise they want to complete in a certain time period.
You can store and calculate ranks with relational database management systems like MySQL on a small scale. However, disk-based databases perform poorly and are prone to scalability issues when million of users' records are involved. The Redis in-memory database server performs optimally for these kinds of operations, and its sorted set data structure can handle a large load efficiently.
In this guide, you'll use the Redis ZSET data type to create a leaderboard with Golang on your Linux server.
Prerequisites
To proceed with this tutorial, ensure you've the following:
1. Create a main.go File
In this tutorial, you'll implement a web application that listens for incoming POST and GET requests to add and retrieve information in a Redis ZSET.
The application will log step count data for different users participating in a fitness tracking competition. In the end, you'll be able to send a curl POST command to add steps for a user and a GET command to list participating members and their rankings.
Begin by creating a project folder for your application. This separates your Golang source code files from the rest of the Linux files to make troubleshooting easier if you encounter errors in the future.
$ mkdir project
Navigate to the new project directory.
$ cd project
Next, open a new main.go file. This file runs the main() function, which fires when you start the application.
$ nano main.go
Then, enter the information below into the main.go file.
package main
import (
"encoding/json"
"fmt"
"net/http"
"github.com/go-redis/redis"
)
func main() {
http.HandleFunc("/scores", httpHandler)
http.ListenAndServe(":8080", nil)
}
func httpHandler(w http.ResponseWriter, req *http.Request) {
redisClient := redis.NewClient(&redis.Options{
Addr: "localhost:6379",
Password: "",
DB: 0,
})
params := map[string]interface{}{}
resp := map[string]interface{}{}
var err error
if req.Method == "GET" {
for k, v := range req.URL.Query() {
params[k] = v[0]
}
resp, err = getScores(redisClient, params)
} else if req.Method == "POST" {
err = json.NewDecoder(req.Body).Decode(¶ms)
resp, err = addScore(redisClient, params)
}
enc := json.NewEncoder(w)
enc.SetIndent("", " ")
if err != nil {
resp = map[string]interface{}{
"error": err.Error(),
}
} else {
if encodingErr := enc.Encode(resp); encodingErr != nil {
fmt.Println("{ error: " + encodingErr.Error() + "}")
}
}
}
Save and close the main.go file when you're through with editing.
In the above file you're importing the encoding/json package to format response data in JSON format. Then, you're using fmt to output any basic response to the user. The package net/http provides HTTP methods to your application while the library github.com/go-redis/redis allows you to communicate to the Redis server.
Under the main() function, you're using the statement http.HandleFunc("/scores", httpHandler) to route incoming requests to the httpHandler function. Then, you're starting a web server on port 8080 using the statement http.ListenAndServe(":8080", nil)
In the httpHandler function, you're connecting to the Redis server using the statement redisClient := redis.NewClient().... Next, you're creating a Golang map of [string]interface{} by retrieving GET and POST variables from the req.URL.Query() and req.Body functions.
Then, you're using the Golang if {...} else {...} statement to forward incoming requests to either an addScore(redisClient, params) function or a getScores(redisClient, params) function. In the next, step you'll create the addScore and getScores functions in separate files.
2. Create an add_score.go File
In the previous main.go function, you're redirecting any POST request to an addScore() function. In this step, you'll create the function insider an add_score.go file.
Use nano to create the add_score.go file.
$ nano add_score.go
Next, enter the information below into the add_score.go file.
package main
import (
"context"
"github.com/go-redis/redis"
)
func addScore(c *redis.Client, p map[string]interface{}) (map[string]interface{}, error) {
ctx := context.TODO()
nickname := p["nickname"].(string)
steps := p["steps"].(float64)
//Validate data here in a production environment
err := c.ZAdd(ctx, "app_users", &redis.Z{
Score: steps,
Member: nickname,
}).Err()
if err != nil {
return nil, err
}
rank := c.ZRank(ctx, "app_users", p["nickname"].(string))
if err != nil {
return nil, err
}
response := map[string]interface{}{
"data": map[string]interface{}{
"nickname": p["nickname"].(string),
"rank": rank.Val(),
},
}
return response, nil
}
Save and close the file.
In the above file, you're connecting to the Redis server to add a sorted set entry using the statement c.ZAdd(ctx, "app_users", &redis.Z{ Score: steps, Member: nickname, }).Err(). The app_users is the name of your sorted set as stored in the Redis in-memory database. Then, you're capturing the steps from sample fitness users as a value for the Score variable. You're then distinguishing the different users by populating the Member variable with the different members' nicknames.
Towards the end of the file, you're using the rank := c. ZRank(ctx, "app_users", p["nickname"].(string)) statement to get the rank of the member and then, you're return a map of [string]interface{} to the calling function.
Your add_score.go file primarily handles new entries to the Redis sorted set. To retrieve the entries, you'll create a new file in the next step.
3. Create a get_scores.go File
When working with sorted sets, you can use different Redis functions to compute the members' ranks. For instance, you can return a list of all members with their scores arranged in descending order using the ZRevRangeWithScores() function. Also, you can count total entries in a sorted set using the function ZCount().
In this step, you'll create a file that uses the two functions to retrieve and return members' scores from the Redis server.
Create the get_scores.go File.
$ nano get_scores.go
Enter the information below into the get_scores.go file.
package main
import (
"context"
"fmt"
"strconv"
"github.com/go-redis/redis"
)
func getScores(c *redis.Client, p map[string]interface{}) (map[string]interface{}, error) {
ctx := context.TODO()
start, err := strconv.ParseInt(fmt.Sprint(p["start"]), 10, 64)
if err != nil {
return nil, err
}
stop, err := strconv.ParseInt(fmt.Sprint(p["stop"]), 10, 64)
if err != nil {
return nil, err
}
total, err := c.ZCount(ctx, "app_users", "-inf", "+inf").Result() //int64
if err != nil {
return nil, err
}
scores, err := c.ZRevRangeWithScores(ctx, "app_users", start, stop).Result() //highest to lowest score
if err != nil {
return nil, err
}
data := []map[string]interface{}{}
for _, z := range scores {
record := map[string]interface{}{}
rank := c.ZRank(ctx, "app_users", z.Member.(string))
if err != nil {
return nil, err
}
record["nickname"] = z.Member.(string)
record["score"] = z.Score
record["rank"] = rank.Val()
data = append(data, record)
}
countPerRequest := stop - start + 1
if stop == -1 {
countPerRequest = total
}
response := map[string]interface{}{
"data": data,
"meta": map[string]interface{}{
"start": start,
"stop": stop,
"per_request": countPerRequest,
"total": total,
},
}
return response, nil
}
Save and close the get_scores.go file
In the above file you're parsing the start and stop indices from the URL variables as submitted in a GET request to determine the number of records to return from the Redis set.
Next, you're using the scores, err := c.ZRevRangeWithScores(ctx, "app_users", start, stop).Result() statement to get the members and their respective ranks from the highest to lowest score.
Then, you're looping through the scores array to append members' details to a data map that you're returning to the calling function. In each loop cycle, you're using the rank := c.ZRank(ctx, "app_users", z.Member.(string)) statement to retrieve the rank of each member. In a Redis sorted set, the member with the least score gets the lowest'rank.
Your Redis leaderboard application is now ready for testing.
4. Test the Redis Leaderboard Application
In this step, you'll add and retrieve members' scores from your Redis server by running curl statements against your application's endpoint.
Before you do this, download the Redis package that you're using in your application.
$ go get github.com/go-redis/redis
Next, run the application. The following command starts a web server. Your application should now listen for incoming requests on port 8080. Don't enter any other command on your current SSH terminal.
$ go run ./
Connect to your server in a new terminal window and execute the following curl commands one by one to add entries into the Redis database.
$ curl -X POST localhost:8080/scores -H "Content-Type: application/json" -d '{"nickname": "steven", "steps": 2125}'
$ curl -X POST localhost:8080/scores -H "Content-Type: application/json" -d '{"nickname": "john", "steps": 300}'
$ curl -X POST localhost:8080/scores -H "Content-Type: application/json" -d '{"nickname": "jane", "steps": 1426}'
$ curl -X POST localhost:8080/scores -H "Content-Type: application/json" -d '{"nickname": "francis", "steps": 765}'
$ curl -X POST localhost:8080/scores -H "Content-Type: application/json" -d '{"nickname": "doe", "steps": 923}'
$ curl -X POST localhost:8080/scores -H "Content-Type: application/json" -d '{"nickname": "mary", "steps": 654}'
$ curl -X POST localhost:8080/scores -H "Content-Type: application/json" -d '{"nickname": "mark", "steps": 958}'
$ curl -X POST localhost:8080/scores -H "Content-Type: application/json" -d '{"nickname": "peter", "steps": 1456}'
You should get the following response after executing each POST command above. The JSON response outputs the member's nickname and the rank.
...
{
"data": {
"nickname": "jane",
"rank": 1
}
}
...
Next, run the following GET command to retrieve all records from your application. Set the start and stop indices to 0 and -1 respectively to return all members.
$ curl -X GET "localhost:8080/scores?start=0&stop=-1"
You should now see the JSON response detailing all members, their scores, and ranks. The meta-information displayed at the end shows your set indices(start and stop), total members found in the range of indices(per_request), and all members in the app_users sorted set(total).
{
"data": [
{
"nickname": "steven",
"rank": 7,
"score": 2125
},
{
"nickname": "peter",
"rank": 6,
"score": 1456
},
{
"nickname": "jane",
"rank": 5,
"score": 1426
},
{
"nickname": "mark",
"rank": 4,
"score": 958
},
{
"nickname": "doe",
"rank": 3,
"score": 923
},
{
"nickname": "francis",
"rank": 2,
"score": 765
},
{
"nickname": "mary",
"rank": 1,
"score": 654
},
{
"nickname": "john",
"rank": 0,
"score": 300
}
],
"meta": {
"per_request": 8,
"start": 0,
"stop": -1,
"total": 8
}
}
Next, change the start and stop indices to return only a sub-set from your sorted set. For instance, to retrieve only the first 3 items, run the command below with a start index of 0 and a stop index of 2
$ curl -X GET "localhost:8080/scores?start=0&stop=2"
You should now get 3 records.
{
"data": [
{
"nickname": "steven",
"rank": 7,
"score": 2125
},
{
"nickname": "peter",
"rank": 6,
"score": 1456
},
{
"nickname": "jane",
"rank": 5,
"score": 1426
}
],
"meta": {
"per_request": 3,
"start": 0,
"stop": 2,
"total": 8
}
}
Your application is working as expected. In this tutorial, you enter data manually using the curl command. In a production environment, you should supply data to your application from external data sources such as mobile apps connected to the fitness tracking wrist bands.
Conclusion
In this tutorial, you've created a Redis leaderboard application that returns data in JSON format with Golang on your Linux server. You've used the Redis ZSET functions to add and retrieve the scores of members participating in a fitness tracking application.
Follow the links below to read more Golang tutorials:
