Go程序启动过程的一次追溯

每当编写的Go代码正确执行之后,总是有一种莫名的感觉——成就感。

但是,作为一个志在远方的码农来说,我们不仅要知其然,也要知其所以然。在知道Go代码是怎么编写的情况下,还需要了解Go程序的执行过程中都做了些什么。请跟随我的脚步一起来探索吧。

本文适用人群:对go程序启动过程以及源码感兴趣的小伙伴

运行环境

笔者在整个源码追溯的过程中所依赖的运行环境如下:

// centos 7.9.2009

[root@localhost go-project]# cat /etc/redhat-release

CentOS Linux release 7.9.2009 (Core)

// linux kernal 3.10.0

[root@localhost go-project]# uname -a

Linux localhost.localdomain 3.10.0-1160.el7.x86_64 #1 SMP Mon Oct 19 16:18:59 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

// gdb 7.6.1

[root@localhost go-project]# gdb -v

GNU gdb (GDB) Red Hat Enterprise Linux 7.6.1-120.el7

Copyright (C) 2013 Free Software Foundation, Inc.

License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>

This is free software: you are free to change and redistribute it.

There is NO WARRANTY, to the extent permitted by law. Type "show copying"

and "show warranty" for details.

This GDB was configured as "x86_64-redhat-linux-gnu".

For bug reporting instructions, please see:

<http://www.gnu.org/software/gdb/bugs/>.

// go 1.16.2

// 笔者的go源码位置 /root/go/go1.16.2/

[root@localhost go-project]# go version

go version go1.16.2 linux/amd64

go测试代码

package main

import"fmt"

func main(){

fmt.Println("hello word")

}很简单的一段代码,打印输出 "hello word"。

编译go代码

[root@localhost demo]# go build -gcflags="-N -l" -o main main.go-gcflags为编译时携带的编译参数,用于告知编译器进行某些处理动作。

-N 编译时,禁止优化

-l 编译时,禁止内联通过go build执行之后,得到go的可执行文件main。

使用gdb调试

加载调试文件

[root@localhost demo]# gdb main

GNU gdb (GDB) Red Hat Enterprise Linux 7.6.1-120.el7

Copyright (C) 2013 Free Software Foundation, Inc.

License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>

This is free software: you are free to change and redistribute it.

There is NO WARRANTY, to the extent permitted by law. Type "show copying"

and "show warranty" for details.

This GDB was configured as "x86_64-redhat-linux-gnu".

For bug reporting instructions, please see:

<http://www.gnu.org/software/gdb/bugs/>...

Reading symbols from /root/go-project/demo/main...done.

warning: File "/root/go/go1.16.2/src/runtime/runtime-gdb.py" auto-loading has been declined by your `auto-load safe-path' set to "$debugdir:$datadir/auto-load:/usr/bin/mono-gdb.py".

To enable execution of this file add

add-auto-load-safe-path /root/go/go1.16.2/src/runtime/runtime-gdb.py

line to your configuration file "/root/.gdbinit".

To completely disable this security protection add

set auto-load safe-path /

line to your configuration file "/root/.gdbinit".

For more information about this security protection see the

"Auto-loading safe path" section in the GDB manual. E.g., run from the shell:

info "(gdb)Auto-loading safe path"

(gdb) source /root/go/go1.16.2/src/runtime/runtime-gdb.py

Loading Go Runtime support.其中需要注意的是,gdb识别出来了go源码中用于gdb调试的文件/root/go/go1.16.2/src/runtime/runtime-gdb.py,使用source命令加载进来。

显示go可执行文件调试信息

(gdb) info files

Symbols from "/root/go-project/demo/main".

Local exec file:

`/root/go-project/demo/main', file type elf64-x86-64.

Entry point: 0x465740

0x0000000000401000 - 0x0000000000497773 is .text

0x0000000000498000 - 0x00000000004dbb44 is .rodata

0x00000000004dbce0 - 0x00000000004dc40c is .typelink

0x00000000004dc420 - 0x00000000004dc470 is .itablink

0x00000000004dc470 - 0x00000000004dc470 is .gosymtab

0x00000000004dc480 - 0x0000000000534578 is .gopclntab

0x0000000000535000 - 0x0000000000535020 is .go.buildinfo

0x0000000000535020 - 0x00000000005432e4 is .noptrdata

0x0000000000543300 - 0x000000000054aa90 is .data

0x000000000054aaa0 - 0x00000000005781f0 is .bss

0x0000000000578200 - 0x000000000057d510 is .noptrbss

0x0000000000400f9c - 0x0000000000401000 is .note.go.buildid使用gdb的子命令info,来查看目标文件的调试信息。

(gdb) help

info files -- Names of targets and files being debugged

可以看到可执行文件/root/go-project/demo/main的如下信息:

file type elf64-x86-64 是64位ELF(Linux可执行文件格式)格式的文件Entry point: 0x465740 程序入口地址是 0x465740可执行文件的各个段信息以及虚拟内存地址位置信息

通过入口地址追溯go启动过程

通过打断点的方式,来找对应的方法调用过程。

(gdb) b *0x465740

Breakpoint 1 at 0x465740: file /root/go/go1.16.2/src/runtime/rt0_linux_amd64.s, line 8.找到入口位置在 /root/go/go1.16.2/src/runtime/rt0_linux_amd64.s 的第8行

[root@localhost demo]# vim /root/go/go1.16.2/src/runtime/rt0_linux_amd64.s +8

## /root/go/go1.16.2/src/runtime/rt0_linux_amd64.s 文件

1 // Copyright 2009 The Go Authors. All rights reserved.

2 // Use of this source code is governed by a BSD-style

3 // license that can be found in the LICENSE file.

4

5 #include "textflag.h"

6

7 TEXT _rt0_amd64_linux(SB),NOSPLIT,$-8

8 JMP _rt0_amd64(SB)发现跳到了_rt0_amd64方法中,继续追。

(gdb) b _rt0_amd64

Breakpoint 2 at 0x4621a0: file /root/go/go1.16.2/src/runtime/asm_amd64.s, line 15.打开_rt0_amd64所在/root/go/go1.16.2/src/runtime/asm_amd64.s文件,找到对应逻辑。

[root@localhost demo]# vim /root/go/go1.16.2/src/runtime/asm_amd64.s +15

1 // Copyright 2009 The Go Authors. All rights reserved.

2 // Use of this source code is governed by a BSD-style

3 // license that can be found in the LICENSE file.

4

5 #include "go_asm.h"

6 #include "go_tls.h"

7 #include "funcdata.h"

8 #include "textflag.h"

9

10 // _rt0_amd64 is common startup code for most amd64 systems when using

11 // internal linking. This is the entry point for the program from the

12 // kernel for an ordinary -buildmode=exe program. The stack holds the

13 // number of arguments and the C-style argv.

14 TEXT _rt0_amd64(SB),NOSPLIT,$-8

15 MOVQ 0(SP), DI // argc

16 LEAQ 8(SP), SI // argv

17 JMP runtime·rt0_go(SB)由注释可知,_rt0_amd64是大多数amd64系统使用时的通用启动代码。在整个逻辑的第三行(即代码17行)又调用了runtime·rt0_go。

继续对runtime·rt0_go打断点,找到对应位置。

此处注意:

Go的汇编是基于Plan9的汇编。其中 runtime·rt0_go 在gdb调试时变为 runtime.rt0_go

注意那一个点的变化 · -> .

如果你想问为什么go的汇编是基于Plan9的汇编?

那么我会告诉我:这帮发明golang的大佬们,当年在贝尔实验室搞出过知名的Unix系统。后来由于某些原因又搞了个plan9系统,可惜plan9系统不怎么知名。大佬或许心有不甘,这不在发明golang语言时,plan9里面的东西终于派上了大用场。

(gdb) b runtime.rt0_go

Breakpoint 3 at 0x4621c0: file /root/go/go1.16.2/src/runtime/asm_amd64.s, line 91.追溯runtime·rt0_go所在文件以及逻辑。

[root@localhost demo]# vim /root/go/go1.16.2/src/runtime/asm_amd64.s +91

87 // Defined as ABIInternal since it does not use the stack-based Go ABI (and

88 // in addition there are no calls to this entry point from Go code).

89 TEXT runtime·rt0_go<ABIInternal>(SB),NOSPLIT,$0

90 // copy arguments forward on an even stack

91 MOVQ DI, AX // argc

92 MOVQ SI, BX // argv

93 SUBQ $(4*8+7), SP // 2args 2auto

94 ANDQ $~15, SP

95 MOVQ AX, 16(SP)

96 MOVQ BX, 24(SP)

97

98 // create istack out of the given (operating system) stack.

99 // _cgo_init may update stackguard.

100 MOVQ $runtime·g0(SB), DI

101 LEAQ (-64*1024+104)(SP), BX

102 MOVQ BX, g_stackguard0(DI)

103 MOVQ BX, g_stackguard1(DI)

104 MOVQ BX, (g_stack+stack_lo)(DI)

105 MOVQ SP, (g_stack+stack_hi)(DI)

106

107 // find out information about the processor we're on

108 MOVL $0, AX

109 CPUID

110 MOVL AX, SI

111 CMPL AX, $0

112 JE nocpuinfo

113

114 // Figure out how to serialize RDTSC.

115 // On Intel processors LFENCE is enough. AMD requires MFENCE.

116 // Don't know about the rest, so let's do MFENCE.

117 CMPL BX, $0x756E6547 // "Genu"

118 JNE notintel

119 CMPL DX, $0x49656E69 // "ineI"

120 JNE notintel

121 CMPL CX, $0x6C65746E // "ntel"

122 JNE notintel

123 MOVB $1, runtime·isIntel(SB)

124 MOVB $1, runtime·lfenceBeforeRdtsc(SB)

125 notintel:

126

127 // Load EAX=1 cpuid flags

128 MOVL $1, AX

129 CPUID

130 MOVL AX, runtime·processorVersionInfo(SB)

131

132 nocpuinfo:

133 // if there is an _cgo_init, call it.

134 MOVQ _cgo_init(SB), AX

135 TESTQ AX, AX

136 JZ needtls

137 // arg 1: g0, already in DI

138 MOVQ $setg_gcc<>(SB), SI // arg 2: setg_gcc

139 #ifdef GOOS_android

140 MOVQ $runtime·tls_g(SB), DX // arg 3: &tls_g

141 // arg 4: TLS base, stored in slot 0 (Android's TLS_SLOT_SELF).

142 // Compensate for tls_g (+16).

143 MOVQ -16(TLS), CX

144 #else

145 MOVQ $0, DX // arg 3, 4: not used when using platform's TLS

146 MOVQ $0, CX

147 #endif

148 #ifdef GOOS_windows

149 // Adjust for the Win64 calling convention.

150 MOVQ CX, R9 // arg 4

151 MOVQ DX, R8 // arg 3

152 MOVQ SI, DX // arg 2

153 MOVQ DI, CX // arg 1

154 #endif

155 CALL AX

156

157 // update stackguard after _cgo_init

158 MOVQ $runtime·g0(SB), CX

159 MOVQ (g_stack+stack_lo)(CX), AX

160 ADDQ $const__StackGuard, AX

161 MOVQ AX, g_stackguard0(CX)

162 MOVQ AX, g_stackguard1(CX)

163

164 #ifndef GOOS_windows

165 JMP ok

166 #endif

167 needtls:

168 #ifdef GOOS_plan9

169 // skip TLS setup on Plan 9

170 JMP ok

171 #endif

172 #ifdef GOOS_solaris

173 // skip TLS setup on Solaris

174 JMP ok

175 #endif

176 #ifdef GOOS_illumos

177 // skip TLS setup on illumos

178 JMP ok

179 #endif

180 #ifdef GOOS_darwin

181 // skip TLS setup on Darwin

182 JMP ok

183 #endif

184 #ifdef GOOS_openbsd

185 // skip TLS setup on OpenBSD

186 JMP ok

187 #endif

188

189 LEAQ runtime·m0+m_tls(SB), DI

190 CALL runtime·settls(SB)

191

192 // store through it, to make sure it works

193 get_tls(BX)

194 MOVQ $0x123, g(BX)

195 MOVQ runtime·m0+m_tls(SB), AX

196 CMPQ AX, $0x123

197 JEQ 2(PC)

198 CALL runtime·abort(SB)

199 ok: // `上面不同的系统最终是跳到了这里`

200 // set the per-goroutine and per-mach "registers"

201 get_tls(BX)

202 LEAQ runtime·g0(SB), CX

203 MOVQ CX, g(BX)

204 LEAQ runtime·m0(SB), AX

205

206 // save m->g0 = g0 `!每个m会有一个用于调度的g0,设置m的g0`

207 MOVQ CX, m_g0(AX)

208 // save m0 to g0->m `g0持有m0的地址`

209 MOVQ AX, g_m(CX)

210

211 CLD // convention is D is always left cleared

212 CALL runtime·check(SB)

213

214 MOVL 16(SP), AX // copy argc

215 MOVL AX, 0(SP)

216 MOVQ 24(SP), AX // copy argv

217 MOVQ AX, 8(SP)

218 CALL runtime·args(SB)

219 CALL runtime·osinit(SB)

220 CALL runtime·schedinit(SB)

221

222 //create a new goroutine to start program `!创建main goroutine 用于执行runtime.main, 见241行注释`

223 MOVQ $runtime·mainPC(SB), AX // entry

224 PUSHQ AX

225 PUSHQ $0 // arg size

226 CALL runtime·newproc(SB)

227 POPQ AX

228 POPQ AX

229

230 // start this M `!让当前线程开始执行 main goroutine`

231 CALL runtime·mstart(SB)

232

233 CALL runtime·abort(SB) // mstart should never return

234 RET

235

236 // Prevent dead-code elimination of debugCallV1, which is

237 // intended to be called by debuggers.

238 MOVQ $runtime·debugCallV1<ABIInternal>(SB), AX

239 RET

240

241 // `mainPC is a function value for runtime.main, to be passed to newproc. `

242 // The reference to runtime.main is made via ABIInternal, since the

243 // actual function (not the ABI0 wrapper) is needed by newproc.

244 DATA runtime·mainPC+0(SB)/8,$runtime·main<ABIInternal>(SB)

245 GLOBL runtime·mainPC(SB),RODATA,$8这段足足有100多行的汇编,其主要作用有以下几点:

根据不同系统初始化寄存器等信息创建m0、g0参数处理、系统、调度初始化调用 runtime.main注意汇编中runtime·rt0_go调用若干方法的所在位置(使用打断点的方式查找):

runtime·check -> /root/go/go1.16.2/src/runtime/runtime1.go, line 137

runtime.args -> /root/go/go1.16.2/src/runtime/runtime1.go, line 61

runtime.osinit -> /root/go/go1.16.2/src/runtime/os_linux.go, line 301

runtime.schedinit -> /root/go/go1.16.2/src/runtime/proc.go, line 600

runtime.main -> /root/go/go1.16.2/src/runtime/proc.go, line 115

runtime.mstart -> /root/go/go1.16.2/src/runtime/proc.go, line 1246

//------------------

// 几个重要的方法

//------------------

// osinit()确定cpu核心数

301 func osinit() {

302 ncpu = getproccount()

......

323 }

// !!!调度器初始化

592// The bootstrap sequence is:

593//

594// call osinit

595// call schedinit

596// make & queue new G

597// call runtime·mstart

598//

599// The new G calls runtime·main.

600 func schedinit() {

601 lockInit(&sched.lock, lockRankSched)

602 lockInit(&sched.sysmonlock, lockRankSysmon)

603 lockInit(&sched.deferlock, lockRankDefer)

604 lockInit(&sched.sudoglock, lockRankSudog)

605 lockInit(&deadlock, lockRankDeadlock)

606 lockInit(&paniclk, lockRankPanic)

607 lockInit(&allglock, lockRankAllg)

608 lockInit(&allpLock, lockRankAllp)

609 lockInit(&reflectOffs.lock, lockRankReflectOffs)

610 lockInit(&finlock, lockRankFin)

611 lockInit(&trace.bufLock, lockRankTraceBuf)

612 lockInit(&trace.stringsLock, lockRankTraceStrings)

613 lockInit(&trace.lock, lockRankTrace)

614 lockInit(&cpuprof.lock, lockRankCpuprof)

615 lockInit(&trace.stackTab.lock, lockRankTraceStackTab)

616// Enforce that this lock is always a leaf lock.

617// All of this lock's critical sections should be

618// extremely short.

619 lockInit(&memstats.heapStats.noPLock, lockRankLeafRank)

620

621// raceinit must be the first call to race detector.

622// In particular, it must be done before mallocinit below calls racemapshadow.

623 _g_ := getg()

624 if raceenabled {

625 _g_.racectx, raceprocctx0 = raceinit()

626 }

627

628 sched.maxmcount = 10000// 最大系统线程数限制为 1万

629

630 // The world starts stopped.

631 worldStopped()

632

633 moduledataverify()

634 stackinit() // 栈初始化

635 mallocinit() // 内存分配器初始化

636 fastrandinit() // must run before mcommoninit

637 mcommoninit(_g_.m, -1)

638 cpuinit() // must run before alginit

639 alginit() // maps must not be used before this call

640 modulesinit() // provides activeModules

641 typelinksinit() // uses maps, activeModules

642 itabsinit() // uses activeModules

643

644 sigsave(&_g_.m.sigmask)

645 initSigmask = _g_.m.sigmask

646

647 goargs() // 处理命令行参数

648 goenvs() // 处理环境变量参数

649 parsedebugvars()

650 gcinit() // 垃圾回收器初始化

651

652 lock(&sched.lock)

653 sched.lastpoll = uint64(nanotime())

654 procs := ncpu // !!!通过cpu core 和 GOMAXPROCS 确定P的数量

655 if n, ok := atoi32(gogetenv("GOMAXPROCS")); ok && n > 0 {

656 procs = n

657 }

658 if procresize(procs) != nil { // 调整P数量

659 throw("unknown runnable goroutine during bootstrap")

660 }

661 unlock(&sched.lock)

662

663// World is effectively started now, as P's can run.

664 worldStarted()

665

666// For cgocheck > 1, we turn on the write barrier at all times

667// and check all pointer writes. We can't do this until after

668// procresize because the write barrier needs a P.

669 if debug.cgocheck > 1 {

670 writeBarrier.cgo = true

671 writeBarrier.enabled = true

672 for _, p := range allp {

673 p.wbBuf.reset()

674 }

675 }

676

677 if buildVersion == "" {

678// Condition should never trigger. This code just serves

679// to ensure runtime·buildVersion is kept in the resulting binary.

680 buildVersion = "unknown"

681 }

682 iflen(modinfo) == 1 {

683// Condition should never trigger. This code just serves

684// to ensure runtime·modinfo is kept in the resulting binary.

685 modinfo = ""

686 }

687 }至此,go程序已经基本启动起来,后面就是执行runtime.main的过程。

追溯runtime.main

runtime.main是用go语言编写的,到这里已经可以不用看汇编了,是不是很兴奋~

114// The main goroutine.

115 func main() {

......

122// Max stack size is 1 GB on 64-bit, 250 MB on 32-bit.

123// Using decimal instead of binary GB and MB because

124// they look nicer in the stack overflow failure message.

125 if sys.PtrSize == 8 { // 设置执行栈的最大限制:64位系统为1G

126 maxstacksize = 1000000000

127 } else {

128 maxstacksize = 250000000// 32位系统为250M

129 }

130

......

139 if GOARCH != "wasm" { // no threads on wasm yet, so no sysmon

140// For runtime_syscall_doAllThreadsSyscall, we

141// register sysmon is not ready for the world to be

142// stopped.

143 atomic.Store(&sched.sysmonStarting, 1)

144 systemstack(func() { // 启动后台监控线程sysmon,sysmon用处可是非常大的哦

145 newm(sysmon, nil, -1)

146 })

147 }

148

......

174 doInit(&runtime_inittask) // 执行runtime包中所有初始化函数init()

......

184 gcenable() // 启动垃圾回收器进行后台操作

......

208 doInit(&main_inittask) // 执行所有用户包中初始化函数init()

......

224 fn := main_main // 执行用户逻辑入口 main.main,就是我们写的那个main()函数

225 fn()

......

252 }至此,go程序已经完全启动起来,并开始执行我们的代码了。

总结

本文基于Linux环境,一步步的追溯go程序启动的大致过程:_rt0_amd64_linux -> _rt0_amd64 -> runtime·rt0_go -> runtime.main -> main.main。

src/runtime/asm_amd64.s -> _rt0_amd64(SB)

src/runtime/asm_amd64.s -> runtime·rt0_go(SB)

src/runtime/sys_linux_amd64.s -> runtime·settls(SB)

src/runtime/runtime1.go -> func check

src/runtime/runtime1.go -> func args

src/runtime/os_linux.go -> func sysargs

src/runtime/os_linux.go -> func osinit

src/runtime/os_linux.go -> func getproccount

src/runtime/os_linux.go -> func sched_getaffinity

src/runtime/os_linux.go -> func getproccount

src/runtime/os_linux.go -> func getHugePageSize

src/runtime/os_linux_x86.go -> func osArchInit

src/runtime/proc.go -> func schedinit

src/runtime/traceback.go -> func tracebackinit

src/runtime/symtab.go -> func moduledataverify

src/runtime/stack.go -> func stackinit

src/runtime/malloc.go -> func mallocinit

src/runtime/proc.go -> func fastrandinit

src/runtime/proc.go -> func mcommoninit

src/runtime/proc.go -> func cpuinit

src/runtime/alg.go -> func alginit

src/runtime/symtab.go -> func modulesinit

src/runtime/type.go -> func typelinksinit

src/runtime/iface.go -> func itabsinit

src/runtime/signal_unix.go -> func msigsave

src/runtime/runtime1.go -> func goargs

src/runtime/os_linux.go -> func goenvs

src/runtime/runtime1.go -> func parsedebugvars

src/runtime/mgc.go -> func gcinit

src/runtime/proc.go -> func procresize

src/runtime/proc.go -> func newproc

src/runtime/stubs.go -> func add

src/runtime/stubs.go -> func getg

src/runtime/stubs.go -> func getcallerpc

src/runtime/stubs.go -> func systemstack

src/runtime/proc.go -> func newproc1

src/runtime/stubs.go -> func getg

src/runtime/runtime1.go -> func acquirem

src/runtime/proc.go -> func gfget

[if no gfree,malg、casgstatus、allgadd]

[if 参数 > 0,memmove 拷贝参数]

src/runtime/stack.go -> func gostartcallfn

src/runtime/sys_x86.go -> func gostartcall

src/runtime/proc.go -> func casgstatus

src/runtime/proc.go -> func runqput

[if npidle!=0 && nmspinning == 0,wakep -> nmspinning+1 -> startm]

src/runtime/runtime1.go -> func releasem

src/runtime/proc.go -> func mstart

src/runtime/proc.go -> func mstart1

src/runtime/proc.go -> func getg

src/runtime/proc.go - func save(getcallerpc(), getcallersp())

src/runtime/stubs.go -> func asminit

src/runtime/stubs.go -> func minit

[if m0,mstartm0]

[if _g_.m == &m0,mstartm0]

[if _g_.m.mstartfn != nil,mstartm0]

[if _g_.m. != &m0,acquirep -> _g_.m.nextp = 0]

src/runtime/proc.go -> func schedule (调度循环)你可能会问:为什么需要学习go程序的启动过程?

笔者想说的是:在日常开发或面试过程中,你可能会听到go的GPM模型、P的总数量由runtime.GOMAXPROCS()控制、init()的初始化过程、go线程最大数量限制、go栈最大限制等等问题时,不止是知道它的存在,而是需要知道它为什么存在以及存在哪里。

所谓:知己知彼百战百胜。这样,在开发过程中才能更得心应手。

参考

《Go语言学习笔记》 雨痕/著

《初识Golang汇编》/《Golang 汇编入门知识总结》- ivansli(笔者整理)

相关文章

学习编程是顺着互联网的发展潮流,是一件好事。新手如何学习...
IT行业是什么工作做什么?IT行业的工作有:产品策划类、页面...
女生学Java好就业吗?女生适合学Java编程吗?目前有不少女生...
Can’t connect to local MySQL server through socket \'/v...
oracle基本命令 一、登录操作 1.管理员登录 # 管理员登录 ...
一、背景 因为项目中需要通北京网络,所以需要连vpn,但是服...