Knowledgebase

Introduction

In this guide, you will learn about the Go programming language, web APIs, and how to build them utilizing Go.

Prerequisites

• Working knowledge of Go.
• Go 1.x installed.
• A text editor to write your code.
• Curl or postman installed to make API requests.

Go

The Go programming language (often referred to as Golang) is one of the fastest-growing programming languages, with its ease of development in building fast, reliable, concurrent, and efficient software. As a result, go has found use in a diversity of applications, including:

• Server-side and cloud-based applications.
• DevOps.
• Systems programming and robotics.
• Command-line tool development.
• Cyber security tool development.

Learn more about Go on the official website.

Web APIs

A web API (Application Programming Interface) is a set of programming code/functionality that allows data transmission between web services and is the most common class of APIs. Web APIs provide transfer functionality between web-based systems, representing a client-server architecture delivering requests from web applications and responses from servers using HTTP (Hypertext Transfer Protocol).

Web APIs can be used to extend the functionality of apps or sites. For instance, Google maps API enables the addition of a map with a company or organization's location. Likewise, Twitter's API offers programmatic access to read and write data which can be used to integrate twitter's functionality into an application, and Youtube's API allows developers to embed youtube videos on their sites or apps.

API protocols/architectures

Protocols are used to standardize data exchange between web services to ensure seamless communication regardless of programming language or operating system used. Examples of API protocols and architectures include:

• Service Object Access Protocol (SOAP) - This is a messaging protocol specification used for exchanging structured information between web services, and it uses XML (Extensible Markup Language) as its messaging format over HTTP; its messages consist of three parts: an envelope, which defines message structure and processing instructions; a set of encoding rules for expressing instances of application-defined databases; and a convention for representing procedure calls and responses. SOAP is mostly used in web-based enterprise software to ensure advanced security of transmitted data and is frequently used for legacy support.
• Remote Procedure Call (RPC) - This is a request-response protocol - in which the client triggers an RPC, that sends a request message to a remote server to execute a specified procedure/function with supplied parameters, and the server sends back a response to the client, and after this exchange - the application continues. The client blocks while the server is processing the request unless an asynchronous request was made to the server by the client e.g an XMLHttpRequest etc. RPCs are intended for relatively simple APIs capable of invoking processes. RPC supports JSON (Javascript Object Notation) and XML formats.
• Representational State Transfer (REST) - This isn't a protocol like the others but an architecture and is the most used for API development. REST is stateless" - in that the API stores no data or status between requests and defines a set of six architectural constraints for building applications that work over HTTP which are client-server architecture, statelessness, cacheability, layered system, code on demand, and a uniform interface. REST is widely considered to be a simpler alternative to SOAP, which many find difficult to use. REST APIs are usually termed RESTful APIs", and work with several message formats, including JSON, XML, HTTP, plain text, etc, although it's mostly used with JSON. It also offers modest security, low bandwidth, and high scalability.
• gRPC - This is a modern open-source high-performance RPC framework that can run in any environment and uses HTTP 2.0 as the underlying transport protocol. It utilizes a binary format called Protocol Buffers to serialize data. gRPC offers very high performance and is lightweight. gRPC is becoming an increasingly popular alternative to REST in building APIs, and you can also expose REST over gRPC.

This article will focus on building a RESTful API in Go using JSON as the message format.

Understanding REST

Each REST API call has a relation between an HTTP verb and the URL - which an API endpoint is mapped to. The resources exposed by an application's REST API are mapped to an API endpoint.

Every REST request should hit the path defined by the URL.

An example of a REST API URL is http://hostname/api_endpoint/query(optional)

REST verbs are HTTP methods supported by the REST architecture, which specify an action to be executed on a specific resource or group of resources. Each request made by a client - sends the following information in the HTTP request:

• REST verb.
• Header information.
• Body (optional).

Due to the stateless nature of REST, there has to be a method to find out whether an operation was successful or not; for this reason, it uses HTTP status codes. REST returns status codes depending on if the operation was executed successfully or not.

Categories of Status Codes

There are over 70+ status codes grouped into categories/families.

• 1xx category (Informational) - This category of status codes beginning with '1' fall under informational codes. Clients must be prepared to accept one or more 1xx codes even if they don't expect them, as unexpected 1xx status responses may be ignored by a user agent.
• 2xx category (Success) - This family of status codes beginning with '2' fall under success codes and indicate that an operation was executed successfully e.g 200, 201, etc.
• 3xx category (Redirection) - These 3xx status codes are used to deliver redirection messages i.e indicate that further action needs to be taken by the user-agent to fulfill the request. These codes could be used in cases where you've moved domains or gone to a new location and you need URL redirection.
• 4xx category (Client Error) - These codes are reserved for cases where errors come from the client-side such as an ill-formed REST method, wrong request format, or trying to access an invalid page. E.g 400, 401, 403, 404, 405, etc.
• 5xx category (Server Error) - These are errors from the server. Although the client request might be perfect, bug in the server can arise. The most commonly used status codes are 500, 501, 502, 503, and 504.

Building a simple RESTful API

We will be building a simple RESTful API that implements CRUD operations on tickets e.g, adding and retrieving ticket information, etc. When designing an API, you first identify resources in your web service and define your API endpoints.

REST verb & endpoint: GET /

• Returns a welcome message

REST verb & endpoint: GET /tickets

• Fetch all tickets in our application

REST verb & endpoint: GET /tickets/[id]

• Fetch the ticket with the corresponding ID

REST verb & endpoint: POST /ticket

• Create a new ticket

REST verb & endpoint: PATCH /tickets/[id]

• Partially update a ticket

REST verb & endpoint: DELETE /tickets/[id]

• Delete a ticket

Note: An API endpoint can be shared across multiple HTTP methods.

These endpoints cover all the operations we need for our API.

Creating our Project

Create a directory inside your $GOPATH/src folder:

$ mkdir $GOPATH/src/go-restful-api

Now, switch to the directory we just created:

$ cd $GOPATH/src/go-restful-api

Installing Gorilla Mux

Gorilla mux is a powerful HTTP router and URL matcher for building Go servers. It implements a request router and dispatcher for matching incoming requests to their respective handlers.

We will use this library to define our API routes and build our server. A route within an API simply defines a specific path to take to get specific data or information as we defined earlier. E.g /tickets is the route to fetch all the tickets in our application.

To install Gorilla Mux, enter the following command in your terminal:

$ go get -u github.com/gorilla/mux

You have successfully installed Gorilla Mux.

Importing Packages

Create a main.go file in our directory:

$ touch main.go

Let's import the necessary packages we need inside the main.go file:

package main

import (
    fmt”
    net/http”
    encoding/json”
    log”
    time”
    strconv”

    github.com/gorilla/mux"
)

We've imported the net/http package, which allows us to build servers and handle requests, along with the encoding/json package for marshaling data into the JSON format, and the github.com/gorilla/mux package for building our router, request handlers, and dispatcher.

Defining our struct

Next, we define a struct for our ticket information:

type ticket struct {
    Id int `json:”id”`
    Owner string `json:”owner”`
    Status string `json:”status”`
}

Note: The json:<key> string - formats the corresponding field of a struct as a JSON key.

Let's add some tickets to our application by using a slice of tickets as our database:

var tickets = []ticket{
    {
        Id: 1234,
        Owner: Doris Sunderland",
        Status: approved",
    },
    {
        Id: 5678,
        Owner: Yvonne Winter",
        Status: pending",
    },
}

Adding route handlers

API endpoints/routes need handler functions that resolve requests made to that specific endpoint. These functions process data from the request and return some information after executing logic. For instance, these handler functions could return HTTP status codes and JSON responses when an API request is made.

Now let's define route handlers for our API endpoints from the table earlier:

GET / handler:

The following function handles GET requests made to the / route of our API:

func home(w http.ResponseWriter, r *http.Request) {
  w.WriteHeader(http.StatusOK)
  fmt.Fprintf(w, "Welcome to our API\n")
}

Handler functions take a http.ResponseWriter and http.Request as parameters. The ResponseWriter is an interface used by an HTTP handler to construct HTTP responses and implements the Header, Write, and WriteHeader methods; while Request represents an HTTP request sent by a client or received by a server.

We used the WriteHeader method to send a http.StatusOK (code 200) response header:

w.WriteHeader(http.StatusOK)

You can also use this method to send different status codes. Next, we write back a welcome message using the ResponseWriter stream:

fmt.Fprintf(w, Welcome to our API\n")

This sends the welcome message to the client when an HTTP request to this API endpoint is sent.

GET /tickets handler:

Let's add the following function to our main.go file to return all the tickets we have in our application.

func fetchAllTickets(w http.ResponseWriter, r *http.Request) {
  w.WriteHeader(http.StatusOK)
  json.NewEncoder(w).Encode(tickets)
}

The json.NewEncoder returns a new encoder that writes values to an output stream. We pass our ResponseWriter object as an argument, then use the Encode method, which accepts an argument and writes its JSON encoding to the stream, followed by a newline.

The following line simply writes the JSON encoding of our tickets to the stream, which is sent back to our client.

json.NewEncoder(w).Encoder(tickets)

GET /tickets/<id> handler:

The following lines of code fetch a ticket with the corresponding id attached to the API endpoint reached. For instance, sending a GET request to the endpoint /tickets/1234" will look for a ticket with an id of 1234 and return it as a JSON output to our client.

func fetchTicket(w http.ResponseWriter, r *http.Request) {
  // mux.Vars returns all path parameters as a map
  id := mux.Vars(r)["id"]
  currentTicketId, _ := strconv.Atoi(id)

  w.Header().Set("Content-Type", "application/json")
  w.WriteHeader(http.StatusOK)

  for _, ticket := range tickets {
    if ticket.Id == currentTicketId {
      json.NewEncoder(w).Encode(ticket)
    }
  }
}

Paths can have variables, and are defined using the format {name} or {name:pattern}. If a regular expression pattern is not defined, the matched variable will be anything until the next slash.

mux.Vars() is a function from the gorilla mux package that takes a http.Request argument and returns all path parameters from the request object as a map. For example, when we register a route - /tickets/{id} where the id is a variable that can take on different values, we access this path variable through the map returned by mux.Vars(r).

So id := mux.Varsr returns the specific id passed in the request.

The line:

currentTicketId, _ := strconv.Atoi(id)

converts the id to an integer. Then we loop through our tickets to find a ticket with that id, and if found - return it as a response.

We also set the content-type field in the header of the response to "application/json":

w.Header().Set("Content-Type", "application/json")

POST /ticket handler:

Let's build a handler that creates a new ticket and adds it to our tickets when a POST request hits the /ticket endpoint.

func createTicket(w http.ResponseWriter, r *http.Request) {
  var newTicket ticket

  if err := json.NewDecoder(r.Body).Decode(&newTicket); err != nil {
      // send an internal server error
      w.WriteHeader(http.StatusInternalServerError)
      fmt.Fprintf(w, "Error parsing JSON request")
      log.Fatal(err)
  }

  tickets = append(tickets, newTicket)

  w.Header().Set("Content-Type", "application/json")
  w.WriteHeader(http.StatusCreated)
  json.NewEncoder(w).Encode(newTicket)
}

We decode the JSON data passed in the body of the request, and parse the data into a newTicket variable of type ticket with:

  if err := json.NewDecoder(r.Body).Decode(&newTicket); err != nil {
      // send an internal server error
      w.WriteHeader(http.StatusInternalServerError)
      fmt.Fprintf(w, "Error parsing JSON request")
      log.Fatal(err)
  }

Then we append it to our list of tickets. We also return a 201 status created code in the header, and the newly created ticket as a response.

Note: This error handling is minimal for brevity. For production use you should return error messages in JSON.

PATCH /tickets/<id> handler:

Next, we build the handler to update a ticket with the corresponding id through a PATCH request.

func updateTicket(w http.ResponseWriter, r *http.Request) {
  id := mux.Vars(r)["id"]
  currentTicketId, _ := strconv.Atoi(id)
  
  var updatedTicket ticket

  w.Header().Set("Content-Type", "application/json")
  if err := json.NewDecoder(r.Body).Decode(&newTicket); err != nil {
      // send an internal server error
      w.WriteHeader(http.StatusInternalServerError)
      fmt.Fprintf(w, "Error parsing JSON request")
      log.Fatal(err)
  }

  for i, ticket := range tickets {
    if ticket.Id == currentTicketId {
          ticket.Owner = updatedTicket.Owner
          ticket.Status = updatedTicket.Status

          tickets[i] = ticket
          json.NewEncoder(w).Encode(ticket)
    }
  }
}

We simply fetch the id in the path, read and decode the body of the request made, and iterate through our list of tickets to find a match and make an update.

DELETE /tickets/<id> handler:

Now to add our final handler, which deletes a ticket with the corresponding id passed as a path parameter.

func deleteTicket(w http.ResponseWriter, r *http.Request) {
  id := mux.Vars(r)["id"]
  currentTicketId, _ := strconv.Atoi(id)
  var tmpTickets []ticket

  for i, ticket := range tickets {
    if ticket.Id == currentTicketId {
      tmpTickets = append(tickets[:i], tickets[i+1:])
      fmt.Fprintf(w, "The ticket with ID %v has been deleted.\n", currentTicketId)
    }
  }
  tickets = tmpTickets
}

We simply iterate through the list of tickets looking for the matching id, append every other entry into a temporary variable (tmpTickets), and reassign the tickets variable the value of the temporary variables, which leaves out the ticket with the matching id - essentially deleting it.

Defining our main function

We have created all the necessary handlers for the endpoints in our API, now let's bring it all together in our main function.

func main() {
  router := mux.NewRouter()
  router.HandleFunc("/", home)
  router.HandleFunc("/tickets", fetchAllTickets).Methods("GET")
  router.HandleFunc("/tickets/{id}", fetchTicket).Methods("GET")
  router.HandleFunc("/ticket", createTicket).Methods("POST")
  router.HandleFunc("/tickets/{id}", updateTicket).Methods("PATCH")
  router.HandleFunc("/tickets/{id}", deleteTicket).Methods("DELETE")

  srv := &http.Server{
    Handler: router,
    Addr: "127.0.0.1:8000",
    // Good practice to enforce timeouts for servers you create!
    WriteTimeout: 15 * time.Second,
    ReadTimeout: 15 * time.Second,
  }
  log.Fatal(srv.ListenAndServe())
}

At the start of our main function, we create a new router using the NewRouter() function from the mux package. This router exposes a dispatcher method - HandlerFunc, which takes two arguments (a string representing our endpoint path and a handler function) and allows us to bind endpoints to their respective handler functions.

The next line:

router.HandlerFunc(/”, home)

maps the /" endpoint to the home handler function we defined earlier.

We can also map specific HTTP methods to our defined routes. This line

router.HandleFunc("/tickets", fetchAllTickets).Methods("GET")

maps GET requests made to the /tickets endpoint to the fetchAllTickets handler. This makes it easier to clearly define what HTTP methods are allowed on different routes.

As paths can have variables like we learned earlier, we specify variables in the path using the {name_of_path_variable} format. To define a variable id in our path, we simply add {id} to our path:

router.HandleFunc("/tickets/{id}", fetchTicket).Methods("GET")

This handles requests to endpoints like /tickets/123", /tickets/456" etc, and saves the second path parameter to the id variable; to access the value of id, we use mux.Varshttp.Request, this returns the value of id in each request e.g in our fetchTicket function handler we accessed id by:

id := mux.Vars(r)["id"]

After registering all our routes with the dispatcher HandlerFunc method, we create a http.Server and pass our router as a handler for the server, along with the IP address and port to host our server. We also enforce timeouts for the server as a recommended practice.

srv := &http.Server{
   Handler: router,
   Addr: "127.0.0.1:8000",
   // Good practice to enforce timeouts for servers you create!
   WriteTimeout: 15 * time.Second,
   ReadTimeout: 15 * time.Second,
}

Next, we host our server with the ListenAndServe() method of http.Server and log errors like this:

log.Fatal(srv.ListenAndServe())

Bringing it all together

If you've followed through with the code, your main.go file should look like this:

package main

import (
  "fmt"
  "net/http"
  "encoding/json"
  "log"
  "time"
  "strconv"

  "github.com/gorilla/mux"
)

type ticket struct {
  Id int `json:"id"`
  Owner string `json:"owner"`
  Status string `json:"status"`
}

var tickets = []ticket{
  {
      Id: 1234,
      Owner: "Doris Sunderland",
      Status: "approved",
  },
  {
      Id: 5678,
      Owner: "Yvonne Winter",
      Status: "pending",
  },
}

func home(w http.ResponseWriter, r *http.Request) {
  w.WriteHeader(http.StatusOK)
  fmt.Fprintf(w, "Welcome to our API\n")
}

func createTicket(w http.ResponseWriter, r *http.Request) {
  var newTicket ticket

  if err := json.NewDecoder(r.Body).Decode(&newTicket); err != nil {
      // send an internal server error
      w.WriteHeader(http.StatusInternalServerError)
      fmt.Fprintf(w, "Error parsing JSON request")
      log.Fatal(err)
  }

  tickets = append(tickets, newTicket)

  w.Header().Set("Content-Type", "application/json")
  w.WriteHeader(http.StatusCreated)
  json.NewEncoder(w).Encode(newTicket)
}

func fetchTicket(w http.ResponseWriter, r *http.Request) {
  // mux.Vars returns all path parameters as a map
  id := mux.Vars(r)["id"]
  currentTicketId, _ := strconv.Atoi(id)

  w.Header().Set("Content-Type", "application/json")
  w.WriteHeader(http.StatusOK)

  for _, ticket := range tickets {
    if ticket.Id == currentTicketId {
      json.NewEncoder(w).Encode(ticket)
    }
  }
}

func fetchAllTickets(w http.ResponseWriter, r *http.Request) {
  json.NewEncoder(w).Encode(tickets)
}

func updateTicket(w http.ResponseWriter, r *http.Request) {
  id := mux.Vars(r)["id"]
  currentTicketId, _ := strconv.Atoi(id)
  
  var updatedTicket ticket

  w.Header().Set("Content-Type", "application/json")
  if err := json.NewDecoder(r.Body).Decode(&newTicket); err != nil {
      // send an internal server error
      w.WriteHeader(http.StatusInternalServerError)
      fmt.Fprintf(w, "Error parsing JSON request")
      log.Fatal(err)
  }

  for i, ticket := range tickets {
    if ticket.Id == currentTicketId {
      ticket.Owner = updatedTicket.Owner
      ticket.Status = updatedTicket.Status

      tickets[i] = ticket
      json.NewEncoder(w).Encode(ticket)
    }
  }
}

func deleteTicket(w http.ResponseWriter, r *http.Request) {
  id := mux.Vars(r)["id"]
  currentTicketId, _ := strconv.Atoi(id)
  var tmpTickets []ticket

  for i, ticket := range tickets {
    if ticket.Id == currentTicketId {
      tmpTickets = append(tickets[:i], tickets[i+1:])
      fmt.Fprintf(w, "The ticket with ID %v has been deleted.\n", currentTicketId)
    }
  }
  tickets = tmpTickets
}

func main() {
  router := mux.NewRouter()
  router.HandleFunc("/", home)
  router.HandleFunc("/tickets", fetchAllTickets).Methods("GET")
  router.HandleFunc("/tickets/{id}", fetchTicket).Methods("GET")
  router.HandleFunc("/ticket", createTicket).Methods("POST")
  router.HandleFunc("/tickets/{id}", updateTicket).Methods("PATCH")
  router.HandleFunc("/tickets/{id}", deleteTicket).Methods("DELETE")

  srv := &http.Server{
    Handler: router,
    Addr: "127.0.0.1:8000",
    // Good practice to enforce timeouts for servers you create!
    WriteTimeout: 15 * time.Second,
    ReadTimeout: 15 * time.Second,
  }
  log.Fatal(srv.ListenAndServe())
}

We've successfully built our simple ticket RESTful API service, now to make client requests.

Making Requests to the API

Let's run our API server in the terminal:

$ go run $GOPATH/src/go-restful-api/main.go

Open up another terminal to make requests to our API using the curl command-line tool.

$ curl -XGET 127.0.0.1:8000/

This makes a request to root (/") endpoint of our API, and we get back a response:

Welcome to our API

Let's add a ticket. Enter the following line into the terminal:

$ curl -H "Content-Type: application/json" -XPOST 127.0.0.1:8000/ticket -d '{"id":4, "owner":"James Bald", "status":"pending"}'

This makes a POST request to our /ticket endpoint and adds the ticket with the corresponding details to our list of tickets. We get back our just created ticket as a response:

{"id":4,"owner":"James Bald","status":"pending"}

Now let's make a GET request to fetch all the tickets in our application to make sure our recently added ticket is there:

$ curl -H "Content-Type: application/json" -XGET 127.0.0.1:8000/tickets

We get back the following response:

[{"id":1234,"owner":"Doris Sunderland","status":"approved"},{"id":5678,"owner":"Yvonne Winter","status":"pending"},{"id":4,"owner":"James Bald","status":"pending"}]

These are all the tickets in our application, and we can see that our POST request earlier successfully created a ticket and added it to our list.

Let's update the status of ticket id 5678 from pending to approved; to do this we send a PATCH request to the endpoint /tickets/5678 e.g

$ curl -H "Content-Type: application/json" -XPATCH 127.0.0.1:8000/tickets/5678 -d '{"owner":"Yvonne Winter", "status":"approved"}'

We receive the following as a response from the server:

{"id":5678,"owner":"Yvonne Winter","status":"approved"}

To verify that this change happened, we can send a GET request to the /tickets/<id> endpoint to retrieve this specific ticket:

$ curl -H "Content-Type: application/json" -XGET 127.0.0.1:8000/tickets/5678

This returns the JSON response:

{"id":5678,"owner":"Yvonne Winter","status":"approved"}

Now we know our PATCH request was successful.

Let's delete the ticket with id 1234; to do this we send a DELETE request to the /tickets/1234 endpoint e.g

$ curl -H "Content-Type: application/json" -XDELETE 127.0.0.1:8000/tickets/1234

We receive a message back:

The ticket with ID 1234 has been deleted.

Let's make another GET request to view all our tickets:

$ curl -H "Content-Type: application/json" -XGET 127.0.0.1:8000/tickets

We get back the following response:

[{"id":5678,"owner":"Yvonne Winter","status":"approved"},{"id":4,"owner":"James Bald","status":"pending"}]

We can see that the ticket with an id 1234 is absent, meaning our DELETE. Now we've verified that all our API endpoints work as expected.

We have successfully built a simple web API performing CRUD operations in Go using Gorilla mux.

Extending APIs

APIs can be extended to provide extra functionality such as authenticating client API requests, logging of requests made to the server, etc; this can be done using middlewares.

Middlewares are services or entities that sit in between an API request and its handler; they process the request first before passing it off to its handler, basically hooking into a server's request/response processing.

Server Hardening

Server hardening refers to steps that can be taken to mitigate the security risks on a server. To build production grade servers, the following functionality can improve the security and ease troubleshooting on host servers:

• Logging: logging of API requests made to the server aids in troubleshooting of problems, as they provide a means to determine where functionality broke down or something went wrong. Although extremely useful; in the wrong hands, logs could expose the internal workings on an API or web service; as a consequence - logs should be properly handled.
• Encryption: encrypting sensitive information exchanged between clients and server.
• Exposing Ports - all unnecessary ports should be closed, as this might provide attackers with a feasible attack scope.
• Rate Limiting - limiting the amount of client requests within a specific timeframe is called Rate Limiting; this helps reduce strain on our servers as they might get overloaded with requests. Rate limiting also helps mitigate cyber-attacks such as brute forcing, DoS (Denial of Service), and DDoS (Distributed Denial of Service) by putting a cap on the number of requests a client can send at once.

Conclusion

In this article, you have learned about web APIs, API protocols and architectures, and how to build a simple RESTful API using Go.