恐るべし。
Out of Memory Killer
今日初めて見た。
気付かなかったのかもしれないけど、勝手にプロセス殺されるなんて怖いねぇ・・・
でもsshdとかsyslog-ngとかが無差別に対象になったみたいだけど、その後普通に使えているのは何故だろう。
OOM対策なんてできるのかな
無差別という話だけど、やっぱり原因となっているプログラムはきっちりと殺されている。
さすがに原因をほっとくほど愚かじゃないか。
(※ 一応プロセス名は伏せ)
Sep 7 16:46:41 ip-10-130-185-94 kernel: sshd invoked oom-killer: gfp_mask=0x201d2, order=0, oomkilladj=0
Sep 7 16:46:41 ip-10-130-185-94 kernel:
Sep 7 16:46:41 ip-10-130-185-94 kernel: Call Trace:
Sep 7 16:46:41 ip-10-130-185-94 kernel: [] out_of_memory+0x6d/0x1c4
Sep 7 16:46:41 ip-10-130-185-94 kernel: [] __alloc_pages+0x23e/0x2c7
Sep 7 16:46:41 ip-10-130-185-94 kernel: [] __do_page_cache_readahead+0xf4/0x258
Sep 7 16:46:41 ip-10-130-185-94 kernel: [] __alloc_pages+0xf9/0x2c7
Sep 7 16:46:41 ip-10-130-185-94 kernel: [] filemap_nopage+0x140/0x332
Sep 7 16:46:41 ip-10-130-185-94 kernel: [] __handle_mm_fault+0x442/0x114a
Sep 7 16:46:41 ip-10-130-185-94 kernel: [] sched_fork+0xb3/0x123
Sep 7 16:46:41 ip-10-130-185-94 kernel: [] do_page_fault+0xf23/0x128d
Sep 7 16:46:41 ip-10-130-185-94 kernel: [] thread_return+0x0/0x135
Sep 7 16:46:41 ip-10-130-185-94 kernel: [] error_exit+0x0/0x79
Sep 7 16:46:41 ip-10-130-185-94 kernel:
Sep 7 16:46:41 ip-10-130-185-94 kernel: Mem-info:
Sep 7 16:46:41 ip-10-130-185-94 kernel: DMA per-cpu:
Sep 7 16:46:41 ip-10-130-185-94 kernel: CPU 0: Hot: hi: 186, btch: 31 usd: 3 Cold: hi: 62, btch: 15 usd: 52
Sep 7 16:46:41 ip-10-130-185-94 kernel: CPU 1: Hot: hi: 186, btch: 31 usd: 120 Cold: hi: 62, btch: 15 usd: 47
Sep 7 16:46:41 ip-10-130-185-94 kernel: Active:1907187 inactive:62 dirty:0 writeback:0 unstable:0
Sep 7 16:46:41 ip-10-130-185-94 kernel: free:2776 slab:7077 mapped:1 pagetables:6398 bounce:0
Sep 7 16:46:41 ip-10-130-185-94 kernel: DMA free:11104kB min:11144kB low:13928kB high:16716kB active:7628748kB inactive:248kB present:7764880kB pages_scanned:18951818 all_unreclaimable? yes
Sep 7 16:46:41 ip-10-130-185-94 kernel: lowmem_reserve[]: 0 0 0
Sep 7 16:46:41 ip-10-130-185-94 kernel: DMA: 0*4kB 20*8kB 38*16kB 35*32kB 0*64kB 0*128kB 0*256kB 0*512kB 1*1024kB 2*2048kB 1*4096kB = 11104kB
Sep 7 16:46:41 ip-10-130-185-94 kernel: Swap cache: add 0, delete 0, find 0/0, race 0+0
Sep 7 16:46:41 ip-10-130-185-94 kernel: Free swap = 0kB
Sep 7 16:46:41 ip-10-130-185-94 kernel: Total swap = 0kB
Sep 7 16:46:41 ip-10-130-185-94 kernel: Free swap: 0kB
Sep 7 16:46:41 ip-10-130-185-94 kernel: init invoked oom-killer: gfp_mask=0x201d2, order=0, oomkilladj=0
Sep 7 16:46:41 ip-10-130-185-94 kernel:
Sep 7 16:46:41 ip-10-130-185-94 kernel: Call Trace:
Sep 7 16:46:41 ip-10-130-185-94 kernel: [] out_of_memory+0x6d/0x1c4
Sep 7 16:46:41 ip-10-130-185-94 kernel: [] __alloc_pages+0x23e/0x2c7
Sep 7 16:46:41 ip-10-130-185-94 kernel: [] __do_page_cache_readahead+0xf4/0x258
Sep 7 16:46:41 ip-10-130-185-94 kernel: [] __wait_on_bit_lock+0x5b/0x66
Sep 7 16:46:41 ip-10-130-185-94 kernel: [] __lock_page+0x5e/0x64
Sep 7 16:46:41 ip-10-130-185-94 kernel: [] filemap_nopage+0x140/0x332
Sep 7 16:46:41 ip-10-130-185-94 kernel: [] __handle_mm_fault+0x442/0x114a
Sep 7 16:46:41 ip-10-130-185-94 kernel: [] mntput_no_expire+0x1c/0x92
Sep 7 16:46:41 ip-10-130-185-94 kernel: [] do_page_fault+0xf23/0x128d
Sep 7 16:46:41 ip-10-130-185-94 kernel: [] error_exit+0x0/0x79
Sep 7 16:46:41 ip-10-130-185-94 kernel:
Sep 7 16:46:41 ip-10-130-185-94 kernel: Mem-info:
Sep 7 16:46:41 ip-10-130-185-94 kernel: DMA per-cpu:
Sep 7 16:46:41 ip-10-130-185-94 kernel: CPU 0: Hot: hi: 186, btch: 31 usd: 3 Cold: hi: 62, btch: 15 usd: 52
Sep 7 16:46:41 ip-10-130-185-94 kernel: CPU 1: Hot: hi: 186, btch: 31 usd: 120 Cold: hi: 62, btch: 15 usd: 47
Sep 7 16:46:41 ip-10-130-185-94 kernel: Active:1907187 inactive:62 dirty:0 writeback:0 unstable:0
Sep 7 16:46:41 ip-10-130-185-94 kernel: free:2776 slab:7077 mapped:1 pagetables:6398 bounce:0
Sep 7 16:46:41 ip-10-130-185-94 kernel: DMA free:11104kB min:11144kB low:13928kB high:16716kB active:7628748kB inactive:248kB present:7764880kB pages_scanned:18980982 all_unreclaimable? yes
Sep 7 16:46:41 ip-10-130-185-94 kernel: lowmem_reserve[]: 0 0 0
Sep 7 16:46:41 ip-10-130-185-94 kernel: DMA: 0*4kB 20*8kB 38*16kB 35*32kB 0*64kB 0*128kB 0*256kB 0*512kB 1*1024kB 2*2048kB 1*4096kB = 11104kB
Sep 7 16:46:41 ip-10-130-185-94 kernel: Swap cache: add 0, delete 0, find 0/0, race 0+0
Sep 7 16:46:41 ip-10-130-185-94 kernel: Free swap = 0kB
Sep 7 16:46:41 ip-10-130-185-94 kernel: Total swap = 0kB
Sep 7 16:46:41 ip-10-130-185-94 kernel: Free swap: 0kB
Sep 7 16:46:41 ip-10-130-185-94 kernel: 1968128 pages of RAM
Sep 7 16:46:41 ip-10-130-185-94 kernel: 38913 reserved pages
Sep 7 16:46:41 ip-10-130-185-94 kernel: 12415 pages shared
Sep 7 16:46:41 ip-10-130-185-94 kernel: 0 pages swap cached
Sep 7 16:46:41 ip-10-130-185-94 kernel: Out of memory: kill process 413 (******) score 2464996 or a child
Sep 7 16:46:41 ip-10-130-185-94 kernel: Killed process 413 (******)
Sep 7 16:46:41 ip-10-130-185-94 kernel: sshd invoked oom-killer: gfp_mask=0x201d2, order=0, oomkilladj=0
Sep 7 16:46:41 ip-10-130-185-94 kernel:
Sep 7 16:46:41 ip-10-130-185-94 kernel: Call Trace:
Sep 7 16:46:41 ip-10-130-185-94 kernel: [] out_of_memory+0x6d/0x1c4
Sep 7 16:46:41 ip-10-130-185-94 kernel: [] __alloc_pages+0x23e/0x2c7
Sep 7 16:46:41 ip-10-130-185-94 kernel: [] __do_page_cache_readahead+0xf4/0x258
Sep 7 16:46:41 ip-10-130-185-94 kernel: [] __alloc_pages+0xf9/0x2c7
Sep 7 16:46:41 ip-10-130-185-94 kernel: [] filemap_nopage+0x140/0x332
Sep 7 16:46:41 ip-10-130-185-94 kernel: [] __handle_mm_fault+0x442/0x114a
Sep 7 16:46:41 ip-10-130-185-94 kernel: [] sched_fork+0xb3/0x123
Sep 7 16:46:41 ip-10-130-185-94 kernel: [] do_page_fault+0xf23/0x128d
Sep 7 16:46:41 ip-10-130-185-94 kernel: [] thread_return+0x0/0x135
Sep 7 16:46:41 ip-10-130-185-94 kernel: [] error_exit+0x0/0x79
Sep 7 16:46:41 ip-10-130-185-94 kernel:
Sep 7 16:46:41 ip-10-130-185-94 kernel: Mem-info:
Sep 7 16:46:41 ip-10-130-185-94 kernel: DMA per-cpu:
Sep 7 16:46:41 ip-10-130-185-94 kernel: CPU 0: Hot: hi: 186, btch: 31 usd: 2 Cold: hi: 62, btch: 15 usd: 52
Sep 7 16:46:41 ip-10-130-185-94 kernel: CPU 1: Hot: hi: 186, btch: 31 usd: 120 Cold: hi: 62, btch: 15 usd: 47
Sep 7 16:46:41 ip-10-130-185-94 kernel: Active:1907173 inactive:62 dirty:0 writeback:0 unstable:0
Sep 7 16:46:41 ip-10-130-185-94 kernel: free:2776 slab:7077 mapped:1 pagetables:6398 bounce:0
Sep 7 16:46:41 ip-10-130-185-94 kernel: DMA free:11104kB min:11144kB low:13928kB high:16716kB active:7628692kB inactive:248kB present:7764880kB pages_scanned:19012072 all_unreclaimable? yes
Sep 7 16:46:41 ip-10-130-185-94 kernel: lowmem_reserve[]: 0 0 0
Sep 7 16:46:41 ip-10-130-185-94 kernel: DMA: 0*4kB 20*8kB 38*16kB 35*32kB 0*64kB 0*128kB 0*256kB 0*512kB 1*1024kB 2*2048kB 1*4096kB = 11104kB
Sep 7 16:46:41 ip-10-130-185-94 kernel: Swap cache: add 0, delete 0, find 0/0, race 0+0
Sep 7 16:46:41 ip-10-130-185-94 kernel: Free swap = 0kB
Sep 7 16:46:41 ip-10-130-185-94 kernel: Total swap = 0kB
Sep 7 16:46:41 ip-10-130-185-94 kernel: Free swap: 0kB
Sep 7 16:46:41 ip-10-130-185-94 kernel: 1968128 pages of RAM
Sep 7 16:46:41 ip-10-130-185-94 kernel: 38913 reserved pages
Sep 7 16:46:41 ip-10-130-185-94 kernel: 12415 pages shared
Sep 7 16:46:41 ip-10-130-185-94 kernel: 0 pages swap cached
Sep 7 16:46:41 ip-10-130-185-94 kernel: Out of memory: kill process 20714 (******) score 308124 or a child
Sep 7 16:46:41 ip-10-130-185-94 kernel: Killed process 20714 (******)
Sep 7 16:46:41 ip-10-130-185-94 kernel: init invoked oom-killer: gfp_mask=0x201d2, order=0, oomkilladj=0
Sep 7 16:46:41 ip-10-130-185-94 kernel:
Sep 7 16:46:41 ip-10-130-185-94 kernel: Call Trace:
Sep 7 16:46:41 ip-10-130-185-94 kernel: [] out_of_memory+0x6d/0x1c4
Sep 7 16:46:41 ip-10-130-185-94 kernel: [] __alloc_pages+0x23e/0x2c7
Sep 7 16:46:41 ip-10-130-185-94 kernel: [] __do_page_cache_readahead+0xf4/0x258
Sep 7 16:46:41 ip-10-130-185-94 kernel: [] __wait_on_bit_lock+0x5b/0x66
Sep 7 16:46:41 ip-10-130-185-94 kernel: [] __lock_page+0x5e/0x64
Sep 7 16:46:41 ip-10-130-185-94 kernel: [] filemap_nopage+0x140/0x332
Sep 7 16:46:41 ip-10-130-185-94 kernel: [] __handle_mm_fault+0x442/0x114a
Sep 7 16:46:41 ip-10-130-185-94 kernel: [] mntput_no_expire+0x1c/0x92
Sep 7 16:46:41 ip-10-130-185-94 kernel: [] do_page_fault+0xf23/0x128d
Sep 7 16:46:41 ip-10-130-185-94 kernel: [] error_exit+0x0/0x79
Sep 7 16:46:41 ip-10-130-185-94 kernel:
Sep 7 16:46:41 ip-10-130-185-94 kernel: Mem-info:
Sep 7 16:46:41 ip-10-130-185-94 kernel: DMA per-cpu:
Sep 7 16:46:41 ip-10-130-185-94 kernel: CPU 0: Hot: hi: 186, btch: 31 usd: 2 Cold: hi: 62, btch: 15 usd: 52
Sep 7 16:46:41 ip-10-130-185-94 kernel: CPU 1: Hot: hi: 186, btch: 31 usd: 120 Cold: hi: 62, btch: 15 usd: 47
Sep 7 16:46:41 ip-10-130-185-94 kernel: Active:1907177 inactive:62 dirty:0 writeback:0 unstable:0
Sep 7 16:46:41 ip-10-130-185-94 kernel: free:2776 slab:7077 mapped:1 pagetables:6398 bounce:0
Sep 7 16:46:41 ip-10-130-185-94 kernel: DMA free:11104kB min:11144kB low:13928kB high:16716kB active:7628708kB inactive:248kB present:7764880kB pages_scanned:19023038 all_unreclaimable? yes
Sep 7 16:46:41 ip-10-130-185-94 kernel: lowmem_reserve[]: 0 0 0
Sep 7 16:46:41 ip-10-130-185-94 kernel: DMA: 0*4kB 20*8kB 38*16kB 35*32kB 0*64kB 0*128kB 0*256kB 0*512kB 1*1024kB 2*2048kB 1*4096kB = 11104kB
Sep 7 16:46:41 ip-10-130-185-94 kernel: Swap cache: add 0, delete 0, find 0/0, race 0+0
Sep 7 16:46:41 ip-10-130-185-94 kernel: Free swap = 0kB
Sep 7 16:46:41 ip-10-130-185-94 kernel: Total swap = 0kB
Sep 7 16:46:41 ip-10-130-185-94 kernel: Free swap: 0kB
Sep 7 16:46:41 ip-10-130-185-94 kernel: 1968128 pages of RAM
Sep 7 16:46:41 ip-10-130-185-94 kernel: 38913 reserved pages
Sep 7 16:46:41 ip-10-130-185-94 kernel: 12414 pages shared
Sep 7 16:46:41 ip-10-130-185-94 kernel: 0 pages swap cached
Sep 7 16:46:41 ip-10-130-185-94 kernel: 1968128 pages of RAM
Sep 7 16:46:41 ip-10-130-185-94 kernel: 38913 reserved pages