Gui B.L Software Engineering-Code and posts about tech and programming

How to intercept and mock external API calls in Golang

09 December 2017

Mocking and testing in Golang is easy due to its inherent philosophy of how an interface contract is fulfilled:

"Implementing a Go interface is done implicitly. In Go, there is no need to explicitly implement an interface into a concrete type by specifying any keyword. To implement an interface into a concrete type, just provide the methods with the same signature that is defined in the interface type."

I found myself in need of mocking an external API which would be called by one of my REST apis internally. I ended up doing some research and found 2 possible solutions:

Using Go's Http test lib which I would create a Fake server and pass around a URL struct to be called from inside one of my Worker go routines.
Using HttpRoundTripper which would allow me to "trap" the outbound HTTP calls in order to intercept a specific call and return a JSON mock response. This approach in my opinion would provide a "cleaner" less-intrusive manner to mock my API calls in the sense that I wouldn't have to change my code to adapt it to a specific way for mocking (such as creating additional interfaces and structs and passing variables around).

I also found an existing Github project that uses RoundTripper internally providing a package that can be used from within Go tests. Since I didn't want to reinvent the wheel and after examining the source code of the project I ended up opting to use it for my functional testing approaches.

In this example I'm demonstrating how to mock an external API call.

Sample REST API that calls an external REST API internally

My REST API calls another REST API internally thus for testing purposes I need to mock this external system in order to make the tests "portable" and be able to test my app end-to-end without relying on any external systems being up.

I won't go into details as the code is quite large to fit in a blog POST but here's a snippet of how it calls the external API:

// Searches for offers from external API 
func search(m map[string]string) model.OfferList {
    // try to acquire lock from request Monitor
    if !monitor.IsServiceAvailable(model.externalAPI) {
        log.Printf("Unable to acquire lock from Request Monitor")
        return nil
    }
    // format specific params
    p := filterParams(m)
    endpoint := config.GetProperty("externalEndpoint")
    isKeywordSearch := p["query"] != ""
    page, _ := strconv.ParseInt(p[model.Page], 10, 0)
    apiKey := config.GetProperty("externalApiKey")
    timeout := time.Duration(config.GetIntProperty("defaultTimeout"))  time.Millisecond
    url := endpoint + "/" + config.GetProperty("externalAPISearchPath")
    req, err := http.NewRequest("GET", url, nil)
    req.Header.Set("Accept", "application/json")
    q := req.URL.Query()
    q.Add("format", "json")
    req.URL.RawQuery = q.Encode()
    url = fmt.Sprintf(req.URL.String())
    log.Printf("REST call: %s", url)
    client := &http.Client{
        Timeout: timeout,
    }
    resp, err := client.Do(req)
    if err != nil {
        log.Fatal("Do: ", err)
        return nil
    }
    defer resp.Body.Close()
    var entity SearchResponse
    if err := json.NewDecoder(resp.Body).Decode(&entity); err != nil {
        log.Println(err)
        return nil
    }
    return buildSearchResponse(&entity, pageSize)
// ... and so on

JSON Mock Response

Sample JSON output:

{"list":[{"id":"1234","name":"Better Homes and Gardens","partyName":"apiserver.com" // ... more data

Mock API for testing

Here I use TestMain for starting a local test server and start triggering REST API calls:

package main

import (
    "github.com/guilhebl/go-offer/offer"
    "github.com/stretchr/testify/assert"
    "gopkg.in/jarcoal/httpmock.v1"
    "io/ioutil"
    "log"
    "net/http"
    "net/http/httptest"
    "os"
    "path/filepath"
    "runtime"
    "strings"
    "testing"
)

var app offer.Module

func init() {
    runtime.LockOSThread()
}

func TestMain(m *testing.M) {
    setup()
    go func() {
        exitVal := m.Run()
        teardown()
        os.Exit(exitVal)
    }()
    log.Println("setting up test server...")
    run()
}
func setup() {
    log.Println("SETUP")
}
func teardown() {
    log.Println("TEARDOWN")
}
func check(e error) {
    if e != nil {
        panic(e)
    }
}

func readFile(path string) []byte {
    absPath, _ := filepath.Abs("./" + path)
    dat, err := ioutil.ReadFile(absPath)
    check(err)
    return dat
}

// returns the bytes of a corresponding mock API call for an external resource
func getJsonMockBytes(url string) []byte {
    switch url {
    case "http://api.server.com/search":
        return readFile("api/sample_search_response.json")
    default:
        return nil
    }
}

func executeRequest(req http.Request) httptest.ResponseRecorder {
    rr := httptest.NewRecorder()
    offer.GetInstance().Router.ServeHTTP(rr, req)
    return rr
}

// Tests basic Search that returns trending results from external APIs
func TestSearch(t *testing.T) {
    // register mock for external API endpoint
    httpmock.Activate()
    defer httpmock.DeactivateAndReset()
    httpMethod := "GET"
    apiUrl := "http://api.server.com/search"
    log.Printf("Mocking Search: %s", apiUrl)
    // mock an HTTP 200 OK response
    httpmock.RegisterResponder(httpMethod, apiUrl, httpmock.NewBytesResponder(200, getJsonMockBytes(apiUrl)))
    // call our local server API
    endpoint := "http://localhost:8080/"
    req, _ := http.NewRequest(httpMethod, endpoint, nil)
    response := executeRequest(req)
    assert.Equal(t, 200, response.Code)
    // verify responses
    jsonSnippet := {"list":[{"id":"1234","name":"Better Homes and Gardens","partyName":"apiserver.com"
    body := response.Body.String()
    assert.True(t, strings.HasPrefix(body, jsonSnippet))
    // get the amount of calls for the registered responder
    info := httpmock.GetCallCountInfo()
    count := info[httpMethod+" "+apiUrl]
    assert.Equal(t, 1, count)
    log.Printf("Total External API Calls made: %d", count)
}

compile and run

go test

Source code

Full source code is available here