客户Oracle10.2.0.4 RAC数据库出现ORA-600(kccsbck_first)错误。
详细错误信息为：
Wed Oct 14 17:43:34 2009 ALTER DATABASE FORCE LOGGING Wed Oct 14 17:43:34 2009 ORA-12920 signalled during: ALTER DATABASE FORCE LOGGING... Wed Oct 14 17:43:42 2009 ALTER SYSTEM SET db_unique_name='PRIMARY' SCOPE=SPFILE; Wed Oct 14 17:44:01 2009 ALTER SYSTEM SET log_archive_config='DG_CONFIG=(primary,standby)' SCOPE=SPFILE; Wed Oct 14 17:44:11 2009 ALTER SYSTEM SET log_archive_dest_state_1='ENABLE' SCOPE=SPFILE; Wed Oct 14 17:44:20 2009 ALTER SYSTEM SET log_archive_dest_state_2='ENABLE' SCOPE=SPFILE; Wed Oct 14 17:44:27 2009 ALTER SYSTEM SET remote_login_passwordfile='EXCLUSIVE' SCOPE=SPFILE; Wed Oct 14 17:44:39 2009 ALTER SYSTEM SET log_archive_format='%t_%s_%r.arc' SCOPE=SPFILE; Wed Oct 14 17:44:47 2009 ALTER SYSTEM SET log_archive_max_processes=4 SCOPE=SPFILE; Wed Oct 14 17:44:57 2009 ALTER SYSTEM SET fal_client='PRIMARY' SCOPE=SPFILE; Wed Oct 14 17:45:03 2009 ALTER SYSTEM SET fal_server='STANDBY' SCOPE=SPFILE; Wed Oct 14 17:45:20 2009 ALTER SYSTEM SET standby_file_management='MANUAL' SCOPE=SPFILE; Wed Oct 14 17:45:34 2009 Shutting down instance: further logons disabled Wed Oct 14 17:45:35 2009 Stopping background process CJQ0 Wed Oct 14 17:45:35 2009 Stopping background process MMNL Wed Oct 14 17:45:36 2009 Stopping background process MMON Wed Oct 14 17:45:37 2009 Shutting down instance (immediate) License high water mark = 7 Wed Oct 14 17:45:37 2009 Stopping Job queue slave processes, flags = 7 Wed Oct 14 17:45:37 2009 Job queue slave processes stopped Wed Oct 14 17:45:39 2009 ALTER DATABASE CLOSE NORMAL Wed Oct 14 17:45:39 2009 SMON: disabling tx recovery SMON: disabling cache recovery Wed Oct 14 17:45:41 2009 Shutting down archive processes Archiving IS disabled Wed Oct 14 17:45:46 2009 ARCH shutting down ARC1: Archival stopped Wed Oct 14 17:45:51 2009 ARCH shutting down ARC0: Archival stopped Wed Oct 14 17:45:52 2009 Thread 2 closed at log SEQUENCE 1 Successful close OF redo thread 2 Wed Oct 14 17:45:52 2009 Completed: ALTER DATABASE CLOSE NORMAL Wed Oct 14 17:45:52 2009 ALTER DATABASE DISMOUNT Completed: ALTER DATABASE DISMOUNT ARCH: Archival disabled due TO shutdown: 1089 Shutting down archive processes Archiving IS disabled Archive process shutdown avoided: 0 active ARCH: Archival disabled due TO shutdown: 1089 Shutting down archive processes Archiving IS disabled Archive process shutdown avoided: 0 active Wed Oct 14 17:45:57 2009 freeing rdom 0 Wed Oct 14 17:47:03 2009 Starting ORACLE instance (normal) LICENSE_MAX_SESSION = 0 LICENSE_SESSIONS_WARNING = 0 Interface TYPE 1 lan906 192.168.0.0 configured FROM OCR FOR USE AS a cluster interconnect Interface TYPE 1 lan905 10.200.32.0 configured FROM OCR FOR USE AS a public interface Picked latch-free SCN scheme 3 Autotune OF undo retention IS turned ON. LICENSE_MAX_USERS = 0 SYS auditing IS disabled ksdpec: called FOR event 13740 prior TO event GROUP initialization Starting up ORACLE RDBMS Version: 10.2.0.4.0. System parameters WITH non-DEFAULT VALUES: processes = 600 sessions = 665 timed_statistics = TRUE sga_max_size = 17179869184 lock_sga = TRUE __shared_pool_size = 1644167168 __large_pool_size = 16777216 __java_pool_size = 16777216 __streams_pool_size = 0 spfile = /dev/DATA/ORCL1_spfile sga_target = 12029263872 control_files = /dev/DATA/ORCL1_control01, /dev/DATA/ORCL1_control02, /dev/DATA/ORCL1_control03 control_file_record_keep_time= 15 db_block_size = 8192 __db_cache_size = 10284433408 db_writer_processes = 4 compatible = 10.2.0.3.0 log_archive_config = DG_CONFIG=(PRIMARY,standby) log_archive_dest_1 = LOCATION=/orabak/arch log_archive_dest_state_1 = ENABLE log_archive_dest_state_2 = ENABLE log_archive_max_processes= 4 log_archive_format = %t_%s_%r.arc fal_client = PRIMARY fal_server = STANDBY log_buffer = 52428800 db_file_multiblock_read_count= 32 cluster_database = TRUE cluster_database_instances= 2 standby_file_management = MANUAL thread = 2 fast_start_mttr_target = 600 instance_number = 2 undo_management = AUTO undo_tablespace = UNDOTBS2 undo_retention = 43200 remote_login_passwordfile= EXCLUSIVE db_domain = remote_listener = LISTENERS_ORCL1 job_queue_processes = 10 bitmap_merge_area_size = 1048576 parallel_max_servers = 80 background_dump_dest = /u01/app/oracle/admin/orcl1/bdump user_dump_dest = /u01/app/oracle/admin/orcl1/udump max_dump_file_size = 20971520 core_dump_dest = /u01/app/oracle/admin/orcl1/cdump audit_file_dest = /u01/app/oracle/admin/orcl1/adump hpux_sched_noage = 178 db_name = orcl1 db_unique_name = PRIMARY open_cursors = 1500 pga_aggregate_target = 2147483648 workarea_size_policy = Auto Cluster communication IS configured TO USE the following interface(s) FOR this instance 192.168.0.102 Wed Oct 14 17:47:10 2009 cluster interconnect IPC version:Oracle UDP/IP (generic) IPC Vendor 1 proto 2 PMON started WITH pid=2, OS id=22684 DIAG started WITH pid=4, OS id=22706 PSP0 started WITH pid=6, OS id=22708 LMON started WITH pid=8, OS id=22710 LMD0 started WITH pid=10, OS id=22712 LMS0 started WITH pid=12, OS id=22714 LMS1 started WITH pid=14, OS id=22716 LMS2 started WITH pid=16, OS id=22723 LMS3 started WITH pid=18, OS id=22725 MMAN started WITH pid=20, OS id=22727 DBW0 started WITH pid=22, OS id=22729 DBW1 started WITH pid=24, OS id=22731 DBW2 started WITH pid=26, OS id=22733 DBW3 started WITH pid=28, OS id=22735 LGWR started WITH pid=30, OS id=22737 CKPT started WITH pid=32, OS id=22739 SMON started WITH pid=34, OS id=22741 RECO started WITH pid=36, OS id=22743 CJQ0 started WITH pid=38, OS id=22745 MMON started WITH pid=40, OS id=22747 MMNL started WITH pid=42, OS id=22749 Wed Oct 14 17:47:17 2009 lmon registered WITH NM - instance id 2 (internal mem no 1) Wed Oct 14 17:47:17 2009 Reconfiguration started (OLD inc 0, NEW inc 2) List OF nodes: 1 Global Resource Directory frozen * allocate DOMAIN 0, invalid = TRUE Communication channels reestablished Master broadcasted resource hash VALUE bitmaps Non-LOCAL Process blocks cleaned OUT Wed Oct 14 17:47:17 2009 LMS 1: 0 GCS shadows cancelled, 0 closed Wed Oct 14 17:47:17 2009 LMS 3: 0 GCS shadows cancelled, 0 closed Wed Oct 14 17:47:17 2009 LMS 0: 0 GCS shadows cancelled, 0 closed Wed Oct 14 17:47:17 2009 LMS 2: 0 GCS shadows cancelled, 0 closed SET master node info Submitted ALL remote-enqueue requests Dwn-cvts replayed, VALBLKs dubious ALL grantable enqueues GRANTED Post SMON TO START 1st pass IR Wed Oct 14 17:47:17 2009 LMS 1: 0 GCS shadows traversed, 0 replayed Wed Oct 14 17:47:17 2009 LMS 2: 0 GCS shadows traversed, 0 replayed Wed Oct 14 17:47:17 2009 LMS 3: 0 GCS shadows traversed, 0 replayed Wed Oct 14 17:47:17 2009 LMS 0: 0 GCS shadows traversed, 0 replayed Wed Oct 14 17:47:17 2009 Submitted ALL GCS remote-cache requests Fix WRITE IN gcs resources Reconfiguration complete LCK0 started WITH pid=44, OS id=22761 Wed Oct 14 17:47:18 2009 ALTER DATABASE MOUNT Wed Oct 14 17:47:18 2009 This instance was FIRST TO mount Wed Oct 14 17:47:22 2009 Errors IN file /u01/app/oracle/admin/orcl1/udump/orcl12_ora_22762.trc: ORA-00600: internal error code, arguments: [kccsbck_first], [1], [559446173], [], [], [], [], [] Wed Oct 14 17:47:23 2009 Trace dumping IS performing id=[cdmp_20091014174723] Wed Oct 14 17:47:24 2009 ORA-600 signalled during: ALTER DATABASE MOUNT... Wed Oct 14 17:47:58 2009 Shutting down instance (abort) License high water mark = 1 Instance TERMINATED BY USER, pid = 23014 |
虽然ORA-600错误只是在上面信息的最后几行，但是前面的初始化参数修改和实例重启和这个错误有直接的关系。
很明显，这是一个配置DATA GUARD的过程，不过当前实例在配置DATA GUARD后进行了重启，而这个错误恰好和实例的重启有关。
跟进MOS文档Startup (mount) of 2nd RAC instance fails with ORA-00600 [kccsbck_first] [ID 395156.1]，导致这个错误的原因是两个RAC实例设置的DB_UNIQUE_NAME参数不同所致。前面通过ALTER SYSTEM设置了DB_UNIQUE_NAME，并重启使之生效，而对于另外的节点，由于没有重启系统，DB_UNIQUE_NAME的设置没有改变，因此造成了当前启动的实例与另一个节点上一直运行的实例出现了冲突。
解决这个问题很简单，将另外一个实例也进行重启，就可以解决。