HTTP/2 in GO(三)
相关阅读:
Start
前边两章讲了很多HTTP/2概念性的东西,看起来比较无趣,从这次开始,我们从一些实际用途开始讲起。
本次讲一个非常简单的功能,然后把其内部实现串一下。
这次要实现的功能非常简单,就是一个http2的server,对客户端的请求,只返回一个header信息,并且保持连接,以便在后续任何时候进行一些其他的响应操作。目前看起来这个场景可能没有太大作用,其实HTTP/2做为一个超文本传输协议,目前我们能想到的应用场景还都是普通的web业务,但是老外们的思路就比较广,已经把一些HTTP/2的特性在特定的场景发挥出来了,比如 Amazon的Alexa,Apple的APNS 等。这次实现的这个小功能,就是Alexa里用到的一小部分.
Amazon的avs(Alexa Voice Service)通过HTTP/2实现了全双工的传输功能,其下行功能就用到了这块,Alexa跟avs建立链接后,客户端会发起一个GET /v20160207/directives
的请求,服务端接受请求后,返回一个200的头信息,并hold住链接,后续使用该链接通过Server Push
功能给客户端主动
发送指令。
本次开始,我们先不管Server Push
,先从发送Header
这个小功能开始吧。
HTTP/2在GO语言的实现中没有支持h2c,所以我们必须使用带证书的加密方式,那么首先需要有一张证书。
我们可以使用openssl
自己生成一张:
openssl req -newkey rsa:2048 -nodes -keyout server.key -x509 -days 365 -out server.crt
然后按提示随便输入一些内容就可以得到两个文件,server.key
和server.crt
,其实就是相当于私钥和公钥。当然这个证书是不能在互联网上正常流通使用的,因为证书是自己签发的,没有人给你做担保,能确认这个证书跟它所标识的内容提供方是匹配的。所以我们在做请求测试的时候,需要客户端忽略证书校验才可以。
服务端GO示例的代码如下:
package main
import (
"log"
"net/http"
)
func main() {
http.HandleFunc("/header", func(w http.ResponseWriter, r *http.Request) {
w.Header().Add("X-custom-header", "custom header")
w.WriteHeader(http.StatusNoContent)
if f, ok := w.(http.Flusher); ok {
f.Flush()
}
select {}
})
log.Println("start listen on 8080...")
log.Fatal(http.ListenAndServeTLS(":8080", "server.crt", "server.key", nil))
}
服务运行起来后我们在一个较新的支持HTTP/2的curl命令下执行:
curl "https://localhost:8080/header" -k -i --http2
-k
参数表示忽略证书校验,避免客户端拿到证书后校验不通过而拒绝链接-i
参数表示显示返回的header信息--http2
表示启用http/2,这个参数也可以不带,因为客户端支持的话,会优先使用http/2去链接,服务端不支持的时候降级到http/1.1
这样就实现了只返回了一个header信息,并且链接没有断开。
我们再通过前边介绍过的h2c来看下请求的效果:
可以看到返回的只有一个Header信息,并且是没有END_STREAM
标记的。
本次的实践内容到这里就可以结束了,最终实现的代码很简单,但是为什么这样可以实现呢,在缺少相关资料的情况下,很难知道这样做是可以实现该目的的,那么接下来就从Go语言中对HTTP/2的实现来一探究竟吧:
HTTP/2 Frame in Go
首先来看HTTP/2中的最小传输单元:Frame
:
// A Frame is the base interface implemented by all frame types.
// Callers will generally type-assert the specific frame type:
// *HeadersFrame, *SettingsFrame, *WindowUpdateFrame, etc.
//
// Frames are only valid until the next call to Framer.ReadFrame.
type http2Frame interface {
Header() http2FrameHeader
// invalidate is called by Framer.ReadFrame to make this
// frame's buffers as being invalid, since the subsequent
// frame will reuse them.
invalidate()
}
// A FrameHeader is the 9 byte header of all HTTP/2 frames.
//
// See http://http2.github.io/http2-spec/#FrameHeader
type http2FrameHeader struct {
valid bool // caller can access []byte fields in the Frame
// Type is the 1 byte frame type. There are ten standard frame
// types, but extension frame types may be written by WriteRawFrame
// and will be returned by ReadFrame (as UnknownFrame).
Type http2FrameType
// Flags are the 1 byte of 8 potential bit flags per frame.
// They are specific to the frame type.
Flags http2Flags
// Length is the length of the frame, not including the 9 byte header.
// The maximum size is one byte less than 16MB (uint24), but only
// frames up to 16KB are allowed without peer agreement.
Length uint32
// StreamID is which stream this frame is for. Certain frames
// are not stream-specific, in which case this field is 0.
StreamID uint32
}
// A FrameType is a registered frame type as defined in
// http://http2.github.io/http2-spec/#rfc.section.11.2
type http2FrameType uint8
const (
http2FrameData http2FrameType = 0x0
http2FrameHeaders http2FrameType = 0x1
http2FramePriority http2FrameType = 0x2
http2FrameRSTStream http2FrameType = 0x3
http2FrameSettings http2FrameType = 0x4
http2FramePushPromise http2FrameType = 0x5
http2FramePing http2FrameType = 0x6
http2FrameGoAway http2FrameType = 0x7
http2FrameWindowUpdate http2FrameType = 0x8
http2FrameContinuation http2FrameType = 0x9
)
每个Frame
都包含一个http2FrameHeader
,这个是每个Frame
都有的头信息,在HTTP/2的定义中如下:
+-----------------------------------------------+
| Length (24) |
+---------------+---------------+---------------+
| Type (8) | Flags (8) |
+-+-------------+---------------+-------------------------------+
|R| Stream Identifier (31) |
+=+=============================================================+
| Frame Payload (0...) ...
+---------------------------------------------------------------+
能看到其结构分别对应头信息的一些字段。
然后我们以Headers Frame
为例看下:
// A HeadersFrame is used to open a stream and additionally carries a
// header block fragment.
type http2HeadersFrame struct {
http2FrameHeader
// Priority is set if FlagHeadersPriority is set in the FrameHeader.
Priority http2PriorityParam
headerFragBuf []byte // not owned
}
// PriorityParam are the stream prioritzation parameters.
type http2PriorityParam struct {
// StreamDep is a 31-bit stream identifier for the
// stream that this stream depends on. Zero means no
// dependency.
StreamDep uint32
// Exclusive is whether the dependency is exclusive.
Exclusive bool
// Weight is the stream's zero-indexed weight. It should be
// set together with StreamDep, or neither should be set. Per
// the spec, "Add one to the value to obtain a weight between
// 1 and 256."
Weight uint8
}
+---------------+
|Pad Length? (8)|
+-+-------------+-----------------------------------------------+
|E| Stream Dependency? (31) |
+-+-------------+-----------------------------------------------+
| Weight? (8) |
+-+-------------+-----------------------------------------------+
| Header Block Fragment (*) ...
+---------------------------------------------------------------+
| Padding (*) ...
+---------------------------------------------------------------+
http2PriorityParam
表示了Stream Dependency
和Weight
信息,headerFragBuf
表示 Header Block Fragment
, Padded
信息没有设置单独的结构存储,因为没啥特别的地方会用到,是否存在Pad
信息放在了Frame Header
的Flag
信息里,当Flags.Has(http2FlagHeadersPadded)
时,会取出Pad
的长度,并在取数据时删减掉。
// Frame-specific FrameHeader flag bits.
const (
// ...
// Headers Frame
http2FlagHeadersEndStream http2Flags = 0x1
http2FlagHeadersEndHeaders http2Flags = 0x4
http2FlagHeadersPadded http2Flags = 0x8
http2FlagHeadersPriority http2Flags = 0x20
// ...
)
// 计算Pad的长度
var padLength uint8
if fh.Flags.Has(http2FlagHeadersPadded) {
if p, padLength, err = http2readByte(p); err != nil {
return
}
}
// ...
// 取出 Header Block Fragment
hf.headerFragBuf = p[:len(p)-int(padLength)]
http2Framer
Frame
的读写操作是通过http2Framer
来进行的。
// A Framer reads and writes Frames.
type http2Framer struct {
r io.Reader
lastFrame http2Frame
errDetail error
lastHeaderStream uint32
maxReadSize uint32
headerBuf [http2frameHeaderLen]byte
getReadBuf func(size uint32) []byte
readBuf []byte // cache for default getReadBuf
maxWriteSize uint32 // zero means unlimited; TODO: implement
w io.Writer
wbuf []byte
// ....
}
// http2Framer的操作方法
type http2Framer
func http2NewFramer(w io.Writer, r io.Reader) *http2Framer
func (fr *http2Framer) ErrorDetail() error
func (fr *http2Framer) ReadFrame() (http2Frame, error)
// ...
func (f *http2Framer) WriteData(streamID uint32, endStream bool, data []byte) error
// ...
func (f *http2Framer) WriteHeaders(p http2HeadersFrameParam) error
// ...
func (f *http2Framer) WritePushPromise(p http2PushPromiseParam) error
func (f *http2Framer) WriteRSTStream(streamID uint32, code http2ErrCode) error
// ...
可以看到,通过http2Framer
,我们可以很方便的对http2Frame
进行读写操作,比如http2Framer.ReadFrame
,http2Framer.WritHeaders
等。
http2Framer
是在http2Server.ServeConn
阶段初始化的:
func (s *http2Server) ServeConn(c net.Conn, opts *http2ServeConnOpts) {
baseCtx, cancel := http2serverConnBaseContext(c, opts)
defer cancel()
sc := &http2serverConn{
srv: s,
hs: opts.baseConfig(),
conn: c,
baseCtx: baseCtx,
remoteAddrStr: c.RemoteAddr().String(),
bw: http2newBufferedWriter(c),
// ...
}
// ...
// 将conn交接给http2Framer进行最小粒度的Frame读写.
fr := http2NewFramer(sc.bw, c)
fr.ReadMetaHeaders = hpack.NewDecoder(http2initialHeaderTableSize, nil)
fr.MaxHeaderListSize = sc.maxHeaderListSize()
fr.SetMaxReadFrameSize(s.maxReadFrameSize())
sc.framer = fr
}
然后在serve
阶段通过readFrames()
和writeFrame
进行Frame
的读写操作。
func (sc *http2serverConn) serve() {
// ...
go sc.readFrames() // 读取Frame
// ...
select {
case wr := <-sc.wantWriteFrameCh:
sc.writeFrame(wr) // 写Frame
// ...
}
// ...
}
最后还有一点,就是当我们通过调用了w.Header().Add()
方法设置了Header
之后,如何马上让服务端把这些信息响应到客户端呢,这个时候就是通过Flush()
方法了。
// Optional http.ResponseWriter interfaces implemented.
var (
_ CloseNotifier = (*http2responseWriter)(nil)
_ Flusher = (*http2responseWriter)(nil)
_ http2stringWriter = (*http2responseWriter)(nil)
)
// ...
func (w *http2responseWriter) Flush() {
rws := w.rws
if rws == nil {
panic("Header called after Handler finished")
}
if rws.bw.Buffered() > 0 {
if err := rws.bw.Flush(); err != nil {
// Ignore the error. The frame writer already knows.
return
}
} else {
// The bufio.Writer won't call chunkWriter.Write
// (writeChunk with zero bytes, so we have to do it
// ourselves to force the HTTP response header and/or
// final DATA frame (with END_STREAM) to be sent.
rws.writeChunk(nil)
}
}
// ...
func (rws *http2responseWriterState) writeChunk(p []byte) (n int, err error) {
if !rws.wroteHeader {
rws.writeHeader(200)
}
isHeadResp := rws.req.Method == "HEAD"
if !rws.sentHeader {
// ...
err = rws.conn.writeHeaders(rws.stream, &http2writeResHeaders{
streamID: rws.stream.id,
httpResCode: rws.status,
h: rws.snapHeader,
endStream: endStream,
contentType: ctype,
contentLength: clen,
date: date,
})
}
// ...
}
通过调用Flush()
方法,由于我们没有设置任何body的内容,所以会走到rws.WriteChunk(nil)
逻辑处,这里就是为了在没有内容时,如果希望给客户端响应,来发送Headers Frame
,这里也可以选择在Header Frame
携带END_STREAM
来关闭Stream
,这种是我们在Go中正常响应HEAD
请求时的逻辑,如果我们自己通过Flush
来发送,那么就不会有END_STREAM
,就达到我们的要求了。
ok,至此,整个流程就串起来了。