在10.2.0.4 DATA GUARD的STANDBY数据库中，出现了大量的ORA-1009错误。
错误信息为：
Thu May 31 14:11:16 2012 FAL[client, MRP0]: Error 1009 fetching archived redo log FROM orcl Thu May 31 14:11:16 2012 Errors IN file /opt/app/oradir/admin/orcl_st/bdump/orcl_st_mrp0_23866.trc: ORA-01009: missing mandatory parameter Thu May 31 14:11:46 2012 FAL[client, MRP0]: Error 1009 fetching archived redo log FROM orcl Thu May 31 14:11:46 2012 Errors IN file /opt/app/oradir/admin/orcl_st/bdump/orcl_st_mrp0_23866.trc: ORA-01009: missing mandatory parameter Thu May 31 14:12:16 2012 FAL[client, MRP0]: Error 1009 fetching archived redo log FROM orcl Thu May 31 14:12:16 2012 Errors IN file /opt/app/oradir/admin/orcl_st/bdump/orcl_st_mrp0_23866.trc: ORA-01009: missing mandatory parameter Thu May 31 14:12:46 2012 FAL[client]: Failed TO request gap SEQUENCE GAP - thread 1 SEQUENCE 17433-17449 DBID 1280669543 branch 752888810 FAL[client]: ALL defined FAL servers have been attempted. |
对应的详细TRACE文件为：
*** 2012-05-31 14:11:16.139 Redo shipping client performing standby login *** 2012-05-31 14:11:16.203 66535 kcrr.c Logged ON TO standby successfully Client logon AND security negotiation successful! *** 2012-05-31 14:11:16.208 62692 kcrr.c FAL[client, MRP0]: Error 1009 fetching archived redo log FROM orcl ORA-01009: missing mandatory parameter *** 2012-05-31 14:11:46.233 Redo shipping client performing standby login *** 2012-05-31 14:11:46.316 66535 kcrr.c Logged ON TO standby successfully Client logon AND security negotiation successful! *** 2012-05-31 14:11:46.321 62692 kcrr.c FAL[client, MRP0]: Error 1009 fetching archived redo log FROM orcl ORA-01009: missing mandatory parameter *** 2012-05-31 14:12:16.335 Redo shipping client performing standby login *** 2012-05-31 14:12:16.398 66535 kcrr.c Logged ON TO standby successfully Client logon AND security negotiation successful! *** 2012-05-31 14:12:16.402 62692 kcrr.c FAL[client, MRP0]: Error 1009 fetching archived redo log FROM orcl ORA-01009: missing mandatory parameter *** 2012-05-31 14:12:46.428 |
显然导致问题的原因和FAL的设置有关，查询STANDBY数据库的启动初始化参数，发现只设置了FAL_SERVER而没有设置FAL_CLIENT：
Starting up ORACLE RDBMS Version: 10.2.0.4.0. System parameters WITH non-DEFAULT VALUES: processes = 5000 sessions = 5505 streams_pool_size = 1073741824 nls_date_format = YYYY-MM-DD HH24:MI:SS sga_target = 16290676736 control_files = /home/oracle/setupFiles/stdby.ctl db_block_size = 8192 compatible = 10.2.0.3.0 log_archive_dest_1 = LOCATION=USE_DB_RECOVERY_FILE_DEST VALID_FOR=(ALL_LOGFILES,ALL_ROLES) DB_UNIQUE_NAME=orcl_st log_archive_dest_state_1 = ENABLE log_archive_dest_state_2 = ENABLE log_archive_max_processes= 4 log_archive_format = %t_%s_%r.ARC fal_server = orcl db_file_multiblock_read_count= 16 db_create_file_dest = +DATADG db_recovery_file_dest = +FRADG db_recovery_file_dest_size= 1279157862400 undo_management = AUTO undo_tablespace = UNDOTBS1 undo_retention = 3600 remote_login_passwordfile= EXCLUSIVE db_domain = global_names = TRUE dispatchers = (PROTOCOL=TCP) (SERVICE=orclXDB) utl_file_dir = * job_queue_processes = 0 background_dump_dest = /opt/app/oradir/admin/orcl_st/bdump user_dump_dest = /opt/app/oradir/admin/orcl_st/udump core_dump_dest = /opt/app/oradir/admin/orcl_st/cdump audit_file_dest = /opt/app/oradir/admin/orcl_st/adump open_links = 12 db_name = orcl db_unique_name = orcl_st open_cursors = 5000 pga_aggregate_target = 6509559808 aq_tm_processes = 4 PMON started WITH pid=2, OS id=28406 PSP0 started WITH pid=3, OS id=28408 MMAN started WITH pid=4, OS id=28410 DBW0 started WITH pid=5, OS id=28412 DBW1 started WITH pid=6, OS id=28414 DBW2 started WITH pid=7, OS id=28416 DBW3 started WITH pid=8, OS id=28418 DBW4 started WITH pid=9, OS id=28420 DBW5 started WITH pid=10, OS id=28422 DBW6 started WITH pid=11, OS id=28424 DBW7 started WITH pid=12, OS id=28426 LGWR started WITH pid=13, OS id=28428 CKPT started WITH pid=14, OS id=28430 SMON started WITH pid=15, OS id=28432 RECO started WITH pid=16, OS id=28434 MMON started WITH pid=17, OS id=28436 Thu May 31 13:00:46 2012 starting up 1 dispatcher(s) FOR network address '(ADDRESS=(PARTIAL=YES)(PROTOCOL=TCP))'... |
在印象中FAL_CLIENT的作用似乎并不大，主要是通过FAL_SERVER来确定获取GAP的PRIMARY数据库，莫非缺少CLIENT的设置也会导致异常。查询了MOS，发现果然如此。在FAL request Failed with ORA-01009 on Standby MRP trace. [ID 1329119.1]文章中对这个问题进行了说明，从11.2以后，FAL_CLIENT不再需要设置，但是对应11.2之前的版本，缺少FAL_CLIENT的设置就会导致ORA-1009的错误。
设置FAL_CLIENT参数设置后，错误消失。