dbdao吧 关注:1,186贴子:4,074
  • 3回复贴,共1

Oracle 11.2.0.4 RAC节点重启

只看楼主收藏回复

查看GI和DB的ALERT日志、/var/messages、osw等没发现什么异常。
但oracssdagent_root.log和oracssdmonitor_root.log记录了一些不寻常的信息(如下)。希望吧主和各位高手帮忙分析一下。谢谢!
2017-01-05 16:00:45.266: [ CSSCLNT][3910031104]clssgsHBProcess: Clock shift forward: CSSD TimeOfDay difference 1005, last CSSD TOD epoch seconds 1483603244.191, current
CSSD TOD epoch seconds 1483603245.196
2017-01-05 16:00:45.267: [ CSSCLNT][3910031104]clssgsHBProcess: Clock shift forward: CSSD invariant clock difference 840, last CSSD invariant clock 4161875844 current C
SSD invariant clock 4161876684 CSS shift 165, ms agent shift 65 ms
2017-01-05 16:00:45.267: [ CSSCLNT][3910031104]clssgsHBProcess: Local clock shift 65 ms current TimeOfDay epoch seconds 1483603245.266, last TOD epoch seconds 148360324
4.191, current Invariant Time Clock 4161876854, last ITC 4161875844
2017-01-05 16:00:45.267: [ CSSCLNT][3910031104]clssgsHBProcess: clock shift 65 ms my_slg 4161876854relative CSS ITC 4161876684 last relative CSS ITC 4161875844 differen
ce between relative CSS ITC 840
2017-01-05 16:00:45.360: [ CSSCLNT][3910031104]clssgsHBProcess: Adjusted CSSD TimeOfDay clock 65 ms, from epoch time 1483603245.196 to 1483603245.096 relative CSS ITC 4
161876684
2017-01-05 16:00:46.247: [ CSSCLNT][3910031104]clssgsHBProcess: posting oprocd shift amt -59
2017-01-05 16:00:47.397: [ CSSCLNT][3910031104]clssgsHBProcess: Clock shift forward: CSSD TimeOfDay difference 1095, last CSSD TOD epoch seconds 1483603246.236, current
CSSD TOD epoch seconds 1483603247.331
2017-01-05 16:00:47.397: [ CSSCLNT][3910031104]clssgsHBProcess: Clock shift forward: CSSD invariant clock difference 880, last CSSD invariant clock 4161877884 current C
SSD invariant clock 4161878764 CSS shift 215, ms agent shift 280 ms
2017-01-05 16:00:47.569: [ CSSCLNT][3910031104]clssgsHBProcess: Local clock shift 280 ms current TimeOfDay epoch seconds 1483603247.397, last TOD epoch seconds 14836032
46.247, current Invariant Time Clock 4161878764, last ITC 4161877894


IP属地:福建1楼2017-01-06 20:15回复
    2017-01-05 16:00:47.569: [ CSSCLNT][3910031104]clssgsHBProcess: clock shift 280 ms my_slg 4161878764relative CSS ITC 4161878774 last relative CSS ITC 4161877894 differe
    nce between relative CSS ITC 880
    2017-01-05 16:00:47.569: [ CSSCLNT][3910031104]clssgsHBProcess: Adjusted CSSD TimeOfDay clock to local TOD clock from epoch time 1483603247.331 to 1483603247.397 becasu
    e clock shift 280 ms produced a CSSD TOD clock 0 ms ahead of local TOD, relative CSS ITC 4161878774
    2017-01-05 16:00:48.728: [ CSSCLNT][3910031104]clssgsHBProcess: posting oprocd shift amt -279
    2017-01-05 16:00:54.672: [ CSSCLNT][3910031104]clssgsHBProcess: Clock shift forward: CSSD TimeOfDay difference 1269, last CSSD TOD epoch seconds 1483603252.992, current
    CSSD TOD epoch seconds 1483603254.261
    2017-01-05 16:00:54.672: [ CSSCLNT][3910031104]clssgsHBProcess: Clock shift forward: CSSD invariant clock difference 1210, last CSSD invariant clock 4161884644 current
    CSSD invariant clock 4161885854 CSS shift 59, ms agent shift 0 ms
    2017-01-05 16:00:54.672: [ CSSCLNT][3910031104]clssgsHBProcess: Local clock shift 0 ms current TimeOfDay epoch seconds 1483603254.672, last TOD epoch seconds 1483603254
    .672, current Invariant Time Clock 4161886314, last ITC 4161886314
    2017-01-05 16:00:54.672: [ CSSCLNT][3910031104]clssgsHBProcess: clock shift 0 ms my_slg 4161886314relative CSS ITC 4161887524 last relative CSS ITC 4161886314 differenc
    e between relative CSS ITC 1210
    2017-01-05 16:00:54.673: [ CSSCLNT][3910031104]clssgsHBProcess: Adjusted CSSD TimeOfDay clock to local TOD clock from epoch time 1483603254.261 to 1483603254.672 becasu
    e clock shift 0 ms produced a CSSD TOD clock 0 ms ahead of local TOD, relative CSS ITC 4161887524
    2017-01-05 16:00:55.364: [ CSSCLNT][3910031104]clssgsHBProcess: Clock shift backward: CSSD TimeOfDay difference 692, last CSSD TOD epoch seconds 1483603254.672, current
    CSSD TOD epoch seconds 1483603255.364
    2017-01-05 16:00:55.364: [ CSSCLNT][3910031104]clssgsHBProcess: Clock shift backward: CSSD invariant clock difference 1160, last CSSD invariant clock 4161885854 current
    CSSD invariant clock 4161887014 CSS shift -468, ms agent shift -8 ms
    2017-01-05 16:00:55.364: [ CSSCLNT][3910031104]clssgsHBProcess: Local clock shift -8 ms current TimeOfDay epoch seconds 1483603255.364, last TOD epoch seconds 148360325
    4.672, current Invariant Time Clock 4161887014, last ITC 4161886314
    2017-01-05 16:00:55.365: [ CSSCLNT][3910031104]clssgsHBProcess: clock shift -8 ms my_slg 4161887014relative CSS ITC 4161887474 last relative CSS ITC 4161886314 differen
    ce between relative CSS ITC 1160
    2017-01-05 16:00:55.365: [ CSSCLNT][3910031104]clssgsHBProcess: Adjusted CSSD TimeOfDay clock to local TOD clock from epoch time 1483603255.364 to 1483603255.364 becasu
    e clock shift -8 ms produced a CSSD TOD clock 0 ms ahead of local TOD, relative CSS ITC 4161887474
    2017-01-05 16:01:13.781: [ CSSCLNT][3910031104]clssgsHBProcess: Clock shift backward: CSSD TimeOfDay difference 1000, last CSSD TOD epoch seconds 1483603272.594, curren
    t CSSD TOD epoch seconds 1483603273.594
    2017-01-05 16:01:13.781: [ CSSCLNT][3910031104]clssgsHBProcess: Clock shift backward: CSSD invariant clock difference 610, last CSSD invariant clock 4161904234 current
    CSSD invariant clock 4161904844 CSS shift 390, ms agent shift -3 ms
    2017-01-05 16:01:13.782: [ CSSCLNT][3910031104]clssgsHBProcess: Local clock shift -3 ms current TimeOfDay epoch seconds 1483603273.781, last TOD epoch seconds 148360327
    2.594, current Invariant Time Clock 4161905424, last ITC 4161904234
    2017-01-05 16:01:13.782: [ CSSCLNT][3910031104]clssgsHBProcess: clock shift -3 ms my_slg 4161905424relative CSS ITC 4161904844 last relative CSS ITC 4161904234 differen
    ce between relative CSS ITC 610
    2017-01-05 16:01:13.782: [ CSSCLNT][3910031104]clssgsHBProcess: Adjusted CSSD TimeOfDay clock -3 ms, from epoch time 1483603273.594 to 1483603273.201 relative CSS ITC 4
    161904844
    2017-01-05 16:01:26.789: [ CSSCLNT][3910031104]clssgsHBProcess: Clock shift forward: CSSD TimeOfDay difference 1001, last CSSD TOD epoch seconds 1483603285.609, current
    CSSD TOD epoch seconds 1483603286.610
    2017-01-05 16:01:26.834: [ CSSCLNT][3910031104]clssgsHBProcess: Clock shift forward: CSSD invariant clock difference 580, last CSSD invariant clock 4161917254 current C
    SSD invariant clock 4161917834 CSS shift 421, ms agent shift 554 ms
    2017-01-05 16:01:26.834: [ CSSCLNT][3910031104]clssgsHBProcess: Local clock shift 554 ms current TimeOfDay epoch seconds 1483603286.789, last TOD epoch seconds 14836032
    85.655, current Invariant Time Clock 4161917834, last ITC 4161917254
    2017-01-05 16:01:26.834: [ CSSCLNT][3910031104]clssgsHBProcess: clock shift 554 ms my_slg 4161917834relative CSS ITC 4161917834 last relative CSS ITC 4161917254 differe
    nce between relative CSS ITC 580
    2017-01-05 16:01:26.834: [ CSSCLNT][3910031104]clssgsHBProcess: Adjusted CSSD TimeOfDay clock to local TOD clock from epoch time 1483603286.610 to 1483603286.789 becasu
    e clock shift 554 ms produced a CSSD TOD clock 0 ms ahead of local TOD, relative CSS ITC 4161917834
    2017-01-05 16:01:28.886: [ CSSCLNT][3910031104]clssgsHBProcess: Clock shift forward: CSSD TimeOfDay difference 1000, last CSSD TOD epoch seconds 1483603287.784, current
    CSSD TOD epoch seconds 1483603288.784
    2017-01-05 16:01:28.886: [ CSSCLNT][3910031104]clssgsHBProcess: Clock shift forward: CSSD invariant clock difference 0, last CSSD invariant clock 4161918754 current CSS
    D invariant clock 4161918754 CSS shift 1000, ms agent shift 1102 ms
    2017-01-05 16:01:28.886: [ CSSCLNT][3910031104]clssgsHBProcess: Local clock shift 1102 ms current TimeOfDay epoch seconds 1483603288.886, last TOD epoch seconds 1483603
    287.784, current Invariant Time Clock 4161918754, last ITC 4161918754
    2017-01-05 16:01:28.886: [ CSSCLNT][3910031104]clssgsHBProcess: clock shift 1102 ms my_slg 4161918754relative CSS ITC 4161918754 last relative CSS ITC 4161918754 differ
    ence between relative CSS ITC 0
    2017-01-05 16:01:29.080: [ CSSCLNT][3910031104]clssgsHBProcess: Adjusted CSSD TimeOfDay clock to local TOD clock from epoch time 1483603288.784 to 1483603288.886 becasu
    e clock shift 1102 ms produced a CSSD TOD clock 0 ms ahead of local TOD, relative CSS ITC 4161918754
    2017-01-05 16:01:29.461: [ CSSCLNT][3910031104]clssgsHBProcess: posting oprocd shift amt -1775
    2017-01-05 16:01:32.523: [ CSSCLNT][3910031104]clssgsHBProcess: posting oprocd shift amt -689
    2017-01-05 16:01:35.610: [ AGFW][3930740512] Recvd request to shed the threads
    2017-01-05 16:01:35.675: [CLSFRAME][3930740512] TM [MultiThread] is changing desired thread # to 2. Current # is 3
    2017-01-05 16:01:35.675: [CLSFRAME][3930740512] TM [MultiThread] is changing desired thread # to 4. Current # is 3
    2017-01-05 16:01:56.233: [ CSSCLNT][3910031104]clssgsHBProcess: Clock shift backward: CSSD TimeOfDay difference 1092, last CSSD TOD epoch seconds 1483603315.017, curren
    t CSSD TOD epoch seconds 1483603316.109


    IP属地:福建2楼2017-01-06 20:16
    回复
      2017-01-05 16:01:56.233: [ CSSCLNT][3910031104]clssgsHBProcess: Clock shift backward: CSSD invariant clock difference 1090, last CSSD invariant clock 4161946654 current
      CSSD invariant clock 4161947744 CSS shift 2, ms agent shift -52 ms
      2017-01-05 16:01:56.356: [ CSSCLNT][3910031104]clssgsHBProcess: Local clock shift -52 ms current TimeOfDay epoch seconds 1483603316.233, last TOD epoch seconds 14836033
      15.215, current Invariant Time Clock 4161947874, last ITC 4161946804
      2017-01-05 16:01:56.356: [ CSSCLNT][3910031104]clssgsHBProcess: clock shift -52 ms my_slg 4161947874relative CSS ITC 4161947894 last relative CSS ITC 4161946804 differe
      nce between relative CSS ITC 1090
      2017-01-05 16:01:56.356: [ CSSCLNT][3910031104]clssgsHBProcess: Adjusted CSSD TimeOfDay clock to local TOD clock from epoch time 1483603316.109 to 1483603316.233 becasu
      e clock shift -52 ms produced a CSSD TOD clock 0 ms ahead of local TOD, relative CSS ITC 4161947894
      2017-01-05 16:01:56.356: [ CSSCLNT][3910031104]clssgsHBProcess: posting oprocd shift amt -52
      2017-01-05 16:01:57.159: [ CSSCLNT][3910031104]clssgsHBProcess: Clock shift forward: CSSD TimeOfDay difference 893, last CSSD TOD epoch seconds 1483603316.233, current
      CSSD TOD epoch seconds 1483603317.126
      2017-01-05 16:01:57.159: [ CSSCLNT][3910031104]clssgsHBProcess: Clock shift forward: CSSD invariant clock difference 1020, last CSSD invariant clock 4161947744 current
      CSSD invariant clock 4161948764 CSS shift -127, ms agent shift 6 ms
      2017-01-05 16:01:57.159: [ CSSCLNT][3910031104]clssgsHBProcess: Local clock shift 6 ms current TimeOfDay epoch seconds 1483603317.159, last TOD epoch seconds 1483603316
      .233, current Invariant Time Clock 4161948794, last ITC 4161947874
      2017-01-05 16:01:57.159: [ CSSCLNT][3910031104]clssgsHBProcess: clock shift 6 ms my_slg 4161948794relative CSS ITC 4161948894 last relative CSS ITC 4161947874 differenc
      e between relative CSS ITC 1020
      2017-01-05 16:01:57.201: [ CSSCLNT][3910031104]clssgsHBProcess: Adjusted CSSD TimeOfDay clock to local TOD clock from epoch time 1483603317.126 to 1483603317.159 becasu
      e clock shift 6 ms produced a CSSD TOD clock 0 ms ahead of local TOD, relative CSS ITC 4161948894
      2017-01-05 16:02:01.547: [ CSSCLNT][3910031104]clssgsHBProcess: Clock shift backward: CSSD TimeOfDay difference 1101, last CSSD TOD epoch seconds 1483603320.303, curren
      t CSSD TOD epoch seconds 1483603321.404
      2017-01-05 16:02:01.547: [ CSSCLNT][3910031104]clssgsHBProcess: Clock shift backward: CSSD invariant clock difference 1000, last CSSD invariant clock 4161951944 current
      CSSD invariant clock 4161952944 CSS shift 101, ms agent shift -4 ms
      2017-01-05 16:02:01.547: [ CSSCLNT][3910031104]clssgsHBProcess: Local clock shift -4 ms current TimeOfDay epoch seconds 1483603321.547, last TOD epoch seconds 148360332
      0.381, current Invariant Time Clock 4161953184, last ITC 4161952014
      2017-01-05 16:02:01.672: [ CSSCLNT][3910031104]clssgsHBProcess: clock shift -4 ms my_slg 4161953184relative CSS ITC 4161953014 last relative CSS ITC 4161952014 differen
      ce between relative CSS ITC 1000
      2017-01-05 16:02:01.672: [ CSSCLNT][3910031104]clssgsHBProcess: Adjusted CSSD TimeOfDay clock -4 ms, from epoch time 1483603321.404 to 1483603321.377 relative CSS ITC 4
      161953014
      2017-01-05 16:02:02.468: [ CSSCLNT][3910031104]clssgsHBProcess: Clock shift forward: CSSD TimeOfDay difference 1086, last CSSD TOD epoch seconds 1483603321.377, current
      CSSD TOD epoch seconds 1483603322.463
      2017-01-05 16:02:02.468: [ CSSCLNT][3910031104]clssgsHBProcess: Clock shift forward: CSSD invariant clock difference 1160, last CSSD invariant clock 4161952944 current
      CSSD invariant clock 4161954104 CSS shift -74, ms agent shift 1 ms
      2017-01-05 16:02:02.468: [ CSSCLNT][3910031104]clssgsHBProcess: Local clock shift 1 ms current TimeOfDay epoch seconds 1483603322.468, last TOD epoch seconds 1483603321
      .547, current Invariant Time Clock 4161954104, last ITC 4161953184
      2017-01-05 16:02:02.468: [ CSSCLNT][3910031104]clssgsHBProcess: clock shift 1 ms my_slg 4161954104relative CSS ITC 4161954344 last relative CSS ITC 4161953184 differenc
      e between relative CSS ITC 1160
      2017-01-05 16:02:02.468: [ CSSCLNT][3910031104]clssgsHBProcess: Adjusted CSSD TimeOfDay clock to local TOD clock from epoch time 1483603322.463 to 1483603322.468 becasu
      e clock shift 1 ms produced a CSSD TOD clock 0 ms ahead of local TOD, relative CSS ITC 4161954344
      2017-01-05 16:04:43.905: [ AGENT][3285116704] Logging level for Module: allcomp 0
      2017-01-05 16:04:43.905: [ AGENT][3285116704] Logging level for Module: default 0
      2017-01-05 16:04:43.905: [ AGENT][3285116704] Logging level for Module: AGENT 1
      2017-01-05 16:04:43.905: [ AGENT][3285116704] Logging level for Module: AGFW 1
      2017-01-05 16:04:43.905: [ AGENT][3285116704] Logging level for Module: CLSFRAME 0
      2017-01-05 16:04:43.905: [ AGENT][3285116704] Logging level for Module: CRSCOMM 0
      2017-01-05 16:04:43.905: [ AGENT][3285116704] Logging level for Module: CRSTIMER 0
      2017-01-05 16:04:43.905: [ AGENT][3285116704] Logging level for Module: USRTHRD 1


      IP属地:福建3楼2017-01-06 20:22
      回复
        重启时间是昨天16:04。发现节点2与节点1的系统时间差了42秒。上述提示是不是意味着因为时钟不同步导致节点1重启?


        IP属地:福建4楼2017-01-06 20:25
        回复