使用 curl 检测 HTTP 请求个阶段时间
一个 HTTP 请求涉及到多个阶段
- DNS 域名解析
- 请求从 Client 路由到 Server，Client 与 Server 建立 TCP 链接
- 如果使用了 HTTPS，还涉及 SSL 链接的建立
- Server 准备数据（比如逻辑计算，查询数据库，调用其他后端服务接口）
- Server 返回响应数据
- Client 收到响应数据
- 整个过程还有可能涉及多次重定向
# curl 使用说明
$ man curl
NAME
curl - transfer a URL
SYNOPSIS
curl [options] [URL...]
DESCRIPTION
curl is a tool to transfer data from or to a server, using one of the supported protocols (DICT, FILE, FTP, FTPS, GOPHER, HTTP, HTTPS, IMAP, IMAPS, LDAP, LDAPS, POP3, POP3S, RTMP, RTSP, SCP, SFTP, SMB, SMBS,
SMTP, SMTPS, TELNET and TFTP). The command is designed to work without user interaction.
curl offers a busload of useful tricks like proxy support, user authentication, FTP upload, HTTP post, SSL connections, cookies, file transfer resume, Metalink, and more. As you will see below, the number of
features will make your head spin!
curl is powered by libcurl for all transfer-related features. See libcurl(3) for details.
...
-o, --output <file>
Write output to <file> instead of stdout. If you are using {} or [] to fetch multiple documents, you can use '#' followed by a number in the <file> specifier. That variable will be replaced with the
current string for the URL being fetched. Like in:
curl http://{one,two}.example.com -o "file_#1.txt"
or use several variables like:
curl http://{site,host}.host[1-5].com -o "#1_#2"
You may use this option as many times as the number of URLs you have. For example, if you specify two URLs on the same command line, you can use it like this:
curl -o aa example.com -o bb example.net
and the order of the -o options and the URLs doesn't matter, just that the first -o is for the first URL and so on, so the above command line can also be written as
curl example.com example.net -o aa -o bb
See also the --create-dirs option to create the local directories dynamically. Specifying the output as '-' (a single dash) will force the output to be done to stdout.
See also -O, --remote-name and --remote-name-all and -J, --remote-header-name.
...
-s, --silent
Silent or quiet mode. Don't show progress meter or error messages. Makes Curl mute. It will still output the data you ask for, potentially even to the terminal/stdout unless you redirect it.
...
# 这个参数可以按照用户定义的 format 输出 curl 在请求过程中记录的参数信息
-w, --write-out <format>
Make curl display information on stdout after a completed transfer. The format is a string that may contain plain text mixed with any number of variables. The format can be specified as a literal
"string", or you can have curl read the format from a file with "@filename" and to tell curl to read the format from stdin you write "@-".
The variables present in the output format will be substituted by the value or text that curl thinks fit, as described below. All variables are specified as %{variable_name} and to output a normal %
you just write them as %%. You can output a newline by using \n, a carriage return with \r and a tab space with \t.
NOTE: The %-symbol is a special symbol in the win32-environment, where all occurrences of % must be doubled when using this option.
The variables available are:
content_type The Content-Type of the requested document, if there was any.
filename_effective
The ultimate filename that curl writes out to. This is only meaningful if curl is told to write to a file with the -O, --remote-name or -o, --output option. It's most useful in
combination with the -J, --remote-header-name option. (Added in 7.26.0)
ftp_entry_path The initial path curl ended up in when logging on to the remote FTP server. (Added in 7.15.4)
http_code The numerical response code that was found in the last retrieved HTTP(S) or FTP(s) transfer. In 7.18.2 the alias response_code was added to show the same info.
http_connect The numerical code that was found in the last response (from a proxy) to a curl CONNECT request. (Added in 7.12.4)
http_version The http version that was effectively used. (Added in 7.50.0)
local_ip The IP address of the local end of the most recently done connection - can be either IPv4 or IPv6 (Added in 7.29.0)
local_port The local port number of the most recently done connection (Added in 7.29.0)
num_connects Number of new connects made in the recent transfer. (Added in 7.12.3)
num_redirects Number of redirects that were followed in the request. (Added in 7.12.3)
redirect_url When an HTTP request was made without -L to follow redirects, this variable will show the actual URL a redirect would take you to. (Added in 7.18.2)
remote_ip The remote IP address of the most recently done connection - can be either IPv4 or IPv6 (Added in 7.29.0)
remote_port The remote port number of the most recently done connection (Added in 7.29.0)
scheme The URL scheme (sometimes called protocol) that was effectively used (Added in 7.52.0)
# ==============================================================================================================================
# ========================================= 下面是 curl 在请求过程中统计的 metrics 信息 ==========================================
# ==============================================================================================================================
size_download The total amount of bytes that were downloaded.
size_header The total amount of bytes of the downloaded headers.
size_request The total amount of bytes that were sent in the HTTP request.
size_upload The total amount of bytes that were uploaded.
speed_download The average download speed that curl measured for the complete download. Bytes per second.
speed_upload The average upload speed that curl measured for the complete upload. Bytes per second.
ssl_verify_result
The result of the SSL peer certificate verification that was requested. 0 means the verification was successful. (Added in 7.19.0)
time_appconnect
The time, in seconds, it took from the start until the SSL/SSH/etc connect/handshake to the remote host was completed. (Added in 7.19.0)
time_connect The time, in seconds, it took from the start until the TCP connect to the remote host (or proxy) was completed.
time_namelookup
The time, in seconds, it took from the start until the name resolving was completed.
time_pretransfer
The time, in seconds, it took from the start until the file transfer was just about to begin. This includes all pre-transfer commands and negotiations that are specific to the
particular protocol(s) involved.
time_redirect The time, in seconds, it took for all redirection steps include name lookup, connect, pretransfer and transfer before the final transaction was started. time_redirect shows the complete
execution time for multiple redirections. (Added in 7.12.3)
time_starttransfer
The time, in seconds, it took from the start until the first byte was just about to be transferred. This includes time_pretransfer and also the time the server needed to calculate the
result.
time_total The total time, in seconds, that the full operation lasted. The time will be displayed with millisecond resolution.
url_effective The URL that was fetched last. This is most meaningful if you've told curl to follow location: headers.
Note: If this option is used several times, the last one will be used.
...
使用curl分别计算各个阶段的耗时
$ curl -o /dev/null -s -w '
time_namelookup:\t%{time_namelookup}
time_connect:\t\t%{time_connect}
time_appconnect:\t%{time_appconnect}
time_pretransfer:\t%{time_pretransfer}
time_starttransfer:\t%{time_starttransfer}
time_total:\t\t%{time_total}
time_redirect:\t\t%{time_redirect}
' https://www.baidu.com
time_namelookup: 0.002826 # 从请求开始到域名解析完成耗时
time_connect: 0.039165 # 从请求开始到 TCP 建立完成耗时
time_appconnect: 0.118937 # 从请求开始到应用层（SSL/SSH/etc）连接/握手完成耗时
time_pretransfer: 0.119068 # 从请求开始到准备开始传输的耗时
time_starttransfer: 0.152317 # 从请求开始到开始传输数据（libcurl接收到第一个字节）的耗时
time_total: 0.152444 # 请求总耗时
time_redirect: 0.000000 # 整个请求中重定向的耗时，0 代表没有重定向
各个阶段耗时概要
|
|--namelookup
|--|--connect
|--|--|--appconnect
|--|--|--|--pretransfer
|--|--|--|--|--starttransfer
|--|--|--|--|--|--total
|--|--|--|--|--|--redirect
当
starttransfer - pretransfer耗时比较大时，可能说明 Server 端还有性能优化的空间，这部分通常包括FireWall -> LB -> Service -> Cache/DB，可以具体分析
