33. Requests and forms

So far, a request has only chosen a page. Now the visitor sends you data, in the address or in a form. This chapter reads that data, checks it, and answers with the right status. It also shows the one mistake that a web programmer must never make.

All input is untrusted

Everything in a request comes from outside your program. A visitor can type anything into a form, change the address by hand, or send a request with no browser at all, as you have done with curl.exe. A limit in your HTML, such as a drop-down list, does not limit what can arrive.

So every handler follows the same steps:

1. Read the value. It is always a string.
2. Check it. If it is wrong, reply with an error status and return.
3. Do the work.

This is the validation from Chapter 20, with a status code in place of asking again.

The query string

The part of a URL after ? is the query string. It holds names and values:

http://localhost:8080/add?a=2&b=3

a has the value 2, and b has the value 3, with & between them. r.URL.Query() takes this apart, and .Get reads one value. Here is exercises/33/01-query-string.go:

// Chapter 33 example: reading values from the query string.
// Run with: go run exercises/33/01-query-string.go

package main

import (
    "fmt"
    "net/http"
    "strconv"
)

func handleHello(w http.ResponseWriter, r *http.Request) {
    name := r.URL.Query().Get("name")
    if name == "" {
        http.Error(w, "Please give a name, as in /hello?name=Keiko", http.StatusBadRequest)
        return
    }

    w.Header().Set("Content-Type", "text/plain; charset=utf-8")
    fmt.Fprintf(w, "Hello, %s!\n", name)
}

func handleAdd(w http.ResponseWriter, r *http.Request) {
    query := r.URL.Query()

    a, err := strconv.Atoi(query.Get("a"))
    if err != nil {
        http.Error(w, "a must be a whole number.", http.StatusBadRequest)
        return
    }

    b, err := strconv.Atoi(query.Get("b"))
    if err != nil {
        http.Error(w, "b must be a whole number.", http.StatusBadRequest)
        return
    }

    fmt.Fprintf(w, "%d + %d = %d\n", a, b, a+b)
}

func main() {
    http.HandleFunc("GET /hello", handleHello)
    http.HandleFunc("GET /add", handleAdd)

    fmt.Println("Server running at http://localhost:8080")
    fmt.Println("Press Ctrl+C to stop it.")

    err := http.ListenAndServe("localhost:8080", nil)
    if err != nil {
        fmt.Println("Server stopped:", err)
    }
}

Get returns an empty string for a name that is missing, the same as for an empty value, so the check on name covers both. Numbers need strconv.Atoi and its error check, as in Chapter 21, and bad text becomes a status 400. The visitor made the mistake, so it is a 4xx code. The results:

curl.exe -i "http://localhost:8080/hello?name=Keiko"
HTTP/1.1 200 OK
Hello, Keiko!

curl.exe -i http://localhost:8080/hello
HTTP/1.1 400 Bad Request
Please give a name, as in /hello?name=Keiko

curl.exe -i "http://localhost:8080/add?a=2&b=3"
HTTP/1.1 200 OK
2 + 3 = 5

curl.exe -i "http://localhost:8080/add?a=abc&b=3"
HTTP/1.1 400 Bad Request
a must be a whole number.

A space cannot appear in a URL as it is, so it is written as %20 or +: /hello?name=Keiko%20Ito gives Hello, Keiko Ito!. Get decodes this for you. A browser encodes a URL for you when you type one.

handleHello sets Content-Type to text/plain on purpose. The name came from the visitor, and this says that the reply must not be read as HTML. The section on unsafe output explains why.

Values in the path

A pattern can have a named wildcard in curly braces, which matches one part of the path. r.PathValue reads it. This is exercises/33/02-path-values.go, inside and below the handlers:

func handlePlayer(w http.ResponseWriter, r *http.Request) {
    name := r.PathValue("name")

    w.Header().Set("Content-Type", "text/plain; charset=utf-8")
    fmt.Fprintf(w, "This is the page of %s.\n", name)
}

func handleSquare(w http.ResponseWriter, r *http.Request) {
    n, err := strconv.Atoi(r.PathValue("n"))
    if err != nil {
        http.Error(w, "The path must end in a whole number.", http.StatusBadRequest)
        return
    }
    if n < -1000 || n > 1000 {
        http.Error(w, "The number must be from -1000 to 1000.", http.StatusBadRequest)
        return
    }

    fmt.Fprintf(w, "%d squared is %d\n", n, n*n)
}

func main() {
    http.HandleFunc("GET /players/{name}", handlePlayer)
    http.HandleFunc("GET /square/{n}", handleSquare)

The results:

curl.exe -i http://localhost:8080/players/Keiko
HTTP/1.1 200 OK
This is the page of Keiko.

curl.exe -i http://localhost:8080/players/a/b
HTTP/1.1 404 Not Found
404 page not found

curl.exe -i http://localhost:8080/square/12
HTTP/1.1 200 OK
12 squared is 144

curl.exe -i http://localhost:8080/square/abc
HTTP/1.1 400 Bad Request
The path must end in a whole number.

curl.exe -i http://localhost:8080/square/99999
HTTP/1.1 400 Bad Request
The number must be from -1000 to 1000.

The range check stops the overflow that you saw in the Try this box. {name} matches one part only, so /players/a/b does not match, and /players/ does not either.

Use the path for which thing you want, and the query string for how you want it.

Forms

An HTML form lets a visitor type values, and turns them into a request:

<form action="/message" method="post">
<p><label>Message: <input type="text" name="message"></label></p>
<p><button type="submit">Send</button></p>
</form>

• action is the path that the request goes to.
• method is get or post.
• name on each input is the name of the value. The visitor never sees it, and your handler asks for it by this name.

GET or POST

The method says what the request is for:

• GET asks for something, and changes nothing. A form with method="get" puts its values in the query string, where they can be seen, bookmarked and reloaded. A search box is a good example.
• POST sends something that changes things, such as adding a message. Its values go in the body of the request. A browser asks before sending one twice.

Browsers and other programs may repeat a GET whenever they like, and they may fetch links before you click them. So anything that changes data must be a POST.

Reading a form

r.FormValue("message") reads a value from a form. It looks in the body of a POST and in the query string, so it works for both kinds. Like Get, it returns an empty string for a missing value. Here is exercises/33/03-form-post.go:

// Chapter 33 example: a form, POST, validation, and a redirect.
// Run with: go run exercises/33/03-form-post.go

package main

import (
    "fmt"
    "net/http"
    "strings"
    "unicode/utf8"
)

const maxMessageLength = 100

const formPage = `<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<title>Send a message</title>
</head>
<body>
<h1>Send a message</h1>
<form action="/message" method="post">
<p><label>Message: <input type="text" name="message"></label></p>
<p><button type="submit">Send</button></p>
</form>
</body>
</html>
`

const thanksPage = `<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<title>Thank you</title>
</head>
<body>
<h1>Thank you</h1>
<p>The message arrived. Look at the terminal.</p>
<p><a href="/">Send another</a></p>
</body>
</html>
`

func handleForm(w http.ResponseWriter, r *http.Request) {
    w.Header().Set("Content-Type", "text/html; charset=utf-8")
    fmt.Fprint(w, formPage)
}

func handleMessage(w http.ResponseWriter, r *http.Request) {
    message := strings.TrimSpace(r.FormValue("message"))

    if message == "" {
        http.Error(w, "The message is empty.", http.StatusBadRequest)
        return
    }
    if utf8.RuneCountInString(message) > maxMessageLength {
        http.Error(w, "The message is too long.", http.StatusBadRequest)
        return
    }

    fmt.Printf("Received: %q\n", message)

    http.Redirect(w, r, "/thanks", http.StatusSeeOther)
}

func handleThanks(w http.ResponseWriter, r *http.Request) {
    w.Header().Set("Content-Type", "text/html; charset=utf-8")
    fmt.Fprint(w, thanksPage)
}

func main() {
    http.HandleFunc("GET /{$}", handleForm)
    http.HandleFunc("POST /message", handleMessage)
    http.HandleFunc("GET /thanks", handleThanks)

    fmt.Println("Server running at http://localhost:8080")
    fmt.Println("Press Ctrl+C to stop it.")

    err := http.ListenAndServe("localhost:8080", nil)
    if err != nil {
        fmt.Println("Server stopped:", err)
    }
}

Run it, and open http://localhost:8080. Type a message and send it. The browser ends up on the thanks page, and the terminal shows:

Received: "Hello there"

The handler uses %q on purpose. The text came from a visitor and may contain line endings, and %q keeps it on one line, in quotes.

handleMessage follows the three steps. strings.TrimSpace first, so that a message of only spaces counts as empty. Then the two checks, each ending in return. To count characters and not bytes, it uses utf8.RuneCountInString from Chapter 12. Only then does it do the work, which here is one line of printing.

curl.exe -d sends a POST with the values you give. Here are the three paths through the handler. The long message below is a real run with 101 characters, shortened here:

curl.exe -i -d "message=Hello there" http://localhost:8080/message
HTTP/1.1 303 See Other
Location: /thanks

curl.exe -i -d "message=" http://localhost:8080/message
HTTP/1.1 400 Bad Request
The message is empty.

curl.exe -i -d "message=xxxxx...xxxxx" http://localhost:8080/message
HTTP/1.1 400 Bad Request
The message is too long.

And the wrong method:

curl.exe -i http://localhost:8080/message
HTTP/1.1 405 Method Not Allowed
Allow: POST

Method Not Allowed

Redirect after POST

A successful POST does not send a page. It sends a redirect, which tells the browser to go somewhere else:

http.Redirect(w, r, "/thanks", http.StatusSeeOther)

Status 303, See Other, and the header Location: /thanks make the browser send a new GET /thanks. Why not reply with the thanks page directly? Because of the reload button. Reloading repeats the last request. If the last request was the POST, a reload sends the message again. After a redirect, the last request is the harmless GET, and reloading only shows the thanks page again.

This pattern, POST and then redirect and then GET, is used on nearly every form on the web. Use status 303, and return if any code follows the call.

Forgetting return

http.Error writes the response, but it does not end your function. Leave out the return from handleHello, and the code carries on. A request with no name gets this:

Please give a name, as in /hello?name=Keiko
Hello, !

The status is still 400, but the rest of the handler ran with bad data. In a handler that saves things, it would save them. Every error check in a handler ends in return.

Printing input into HTML is unsafe

Here is the mistake that this chapter promised. It looks harmless. This is exercises/33/04-unsafe-echo.go, with its comment, and it is unsafe on purpose:

func handleEcho(w http.ResponseWriter, r *http.Request) {
    text := r.URL.Query().Get("text")

    w.Header().Set("Content-Type", "text/html; charset=utf-8")
    fmt.Fprintf(w, "<p>You said: %s</p>\n", text)
}

With ?text=hello, the page says You said: hello. Now ask for this, with a piece of HTML as the text. -G puts the value in the query string, and --data-urlencode encodes it:

curl.exe -i -G --data-urlencode "text=<script>alert('hello')</script>" http://localhost:8080/echo

The response body is:

<p>You said: <script>alert('hello')</script></p>

Your program has sent a <script> tag, and a browser runs what is inside it. You can try this in your browser, by opening the address below. A small box should pop up.

http://localhost:8080/echo?text=<script>alert('hello')</script>

In plain words: the visitor has written part of your page. A box is harmless, but a script can do anything that the page can do. Worse, the text does not have to be typed by the person who sees it. Someone can send a friend a link with a script in it. Or, once a server saves messages, one visitor's message could run in the browser of every later visitor.

This is known as cross-site scripting. The fix is to turn < into < and the other special characters into similar codes, so that the browser shows them as text. Doing that by hand, in every handler, is easy to forget, so you will not do it by hand. Chapter 34 uses a package that does it every time.

Until then, reply with text/plain when you print visitor input, as handleHello does. A browser does not run anything in plain text.

Homework

Homework starters are in exercises/33/homework/. Each one runs as it is and replies with a placeholder line.

Stuck or finished? Open the homework solutions page.