Log of events or changes that might affect IGU product quality or delivery. YYYY-MM-DD Event or change ========== ============================================================== 2008-02-07 -- NGS becomes official IGS ACC source for IGU combination products, starting with 1465_4_12 released at 15:00 UTC 2008-03-18 -- ESOC changes analysis system that produces the ESA Ultra- rapid submissions, starting with 1471_2_18 2008-04-02 -- begin parallel ACC product uploads to IGN Data Center, starting with IGU 1473_3_12 2008-04-09 -- begin parallel ACC product uploads to SOPAC Data Center 2008-05-27 -- begin parallel ACC product uploads to KASI Data Center 2008-06-11 -- product delivery to Global Data Centers generally delayed till 2008-06-13 due to ftp connectivity problems, exact nature unknown; problems esp bad to IGN & KASI; IGU 1483_3_18 delayed to CDDIS by ~5:20; 1483_4_12 by ~0:32; 1483_4_18 by ~0:23; 2008-06-12 -- USNO begins submitting improved IGU solutions using absolute antenna calibrations, starting IGU 1483_4_12 2008-06-18 -- restore ESU, GFU, & USU clocks to IGU combination, starting with IGU 1484_3_18; the GFU exclusion was a mistake introduced on 2008-04-27 when GFU was listed as a source of BRDC clocks to avoid problem with combo bombing due to lack of BRDC reference; ESU had been excluded for unknown (and probably longstanding reasons); USU had been excluded due to non- compliance with IGS absolute antenna calibrations, which was corrected on 2008-06-12 2008-06-25 -- fixed bug in ACC combination (distribute_transfered_files.pl) that did not recognize NANU maneuvers since wk 1446_2 (2007-09-23), starting with IGU 1485_4_00; this was almost certainly related to the installation of Bernese 5.0 that day; during that period, maneuvered SVs were not properly rejected from IGUs nor handled fully correctly in IGR or Final combinations 2008-07-14 -- changed IGU ERP rejection limits from DEL_LIMIT: 650 650 1000 1000 120 to DEL_LIMIT: 520 520 700 700 52 (same as Rapids) in file inp/downweight_eop.tab; also changed AC rotation rejection criterion from 5.0 to 1.0 mas in file inp/rejection.tab 2008-07-29 -- fixed bug in weekly IGU comparison reports (e-mailed each Tuesday) that always gave wrong value for PRN32 (= overall RMS for that IGU instead of value for PRN32) 2008-07-30 -- fixed bug in check_nanu_man.pl that caused the 1st IGU combo with a DV manuever during the prediction interval to ignore that event & include a predicted orbit anyway despite claims to the contrary, starting IGU 1490_3_12 2008-07-30 -- delivery of IGU 1490_3_18 delayed by ~0:40 due to process delays caused by failure of NOAA/NOS mail server; mailing process moved to background jobs 2008-08-05 -- exclude ESU & USU clocks in IGU combination, starting with IGU 1491_2_18; recent performance of both has steadily degraded (apparently due mostly to the recently poor stability of the PRN05 clock) so that they are causing the combined clocks to be much worse than with EMU & GFU only; ACs have been contacted for information & ESU/USU have been asked to delete PRN05 (& similarly badly behaved) clocks 2008-08-06 -- begin testing alternative IGU clock referencing strategies to try to avoid timescale misalignment that arises due to using COU BRDC values in the presence of a large satellite clock discontinuity (namely PRN05 on 2008-08-06); nothing seems effective yet 2008-08-08 -- temporarily restored ESU & USU clocks in IGU combination, but switch to using EMU clocks for BRDC timescale instead of COU clocks; this has the undesirable effect of removing the actual EMU clocks from the combination (temporarily, until change in software is implemented) 2008-08-11 -- temporarily return to COU as BRDC timescale reference in order to re-include EMU clocks in combination; in modifying IGU procedures, caused combo at 1492_1_12 to crash & be submitted late by ~40 minutes 2008-08-12 -- modify remote_daily_autocmb to not exclude BRDC reference AC provided that it is listed in "good_clk_cen"; put EMU back as BRDC alignment reference & clock reference center; changes verified in IGU 1492_2_12; initopt_P.tab_ultra now modified (effective 1492_2_18) to: ref_clkcen | emu gfu usu esu zir broadcast_clk_cen | emu gfu usu cou gou brd good_clk_cen | emu gfu usu esu zir exclude_clk_cen | ngu siu jpu usu esu to again exclude ESU & USU clocks due to very poor performance, esp for PRN05 & PRN25; ESU not included in BRDC list due to consistently poorer alignment to GPST 2008-08-18 -- modify remote_daily_autocmb to count a BRDC clock center in nclk if it is also listed in "good_clk_cen"; this is to correct a small bug associated with the mod on 2008-08-12 to include BRDC clock centers in the clock combination if they are good clocks; effective 1493_1_12 2008-08-19 -- fine-tune some IGU website plots 2008-08-20 -- hard-disk filled up on server due to excessive old PPP station obs files; caused IGU 1493_3_00 to fail & _12 to be late; both were resubmitted (with _06 too, even though it was not affected before); disk cleared & procedures changed to remove older PPP directories; login now reports disk availability 2008-08-28 -- IGU 1491_4_06 delayed by ~10 min due to ftp hangup to BKG Data Center to make final check for input AC files; BKG has been having computer problems lately (see IGS Mails #5807, #5812, #5820) 2008-09-05 -- fix bug in getting & using backup AC submissions files when primary data center files are corrupt, to address firewall problem for ESA deposits to CDDIS 2008-09-09 -- changed old GFZ web address in weekly IGU comparison reports to correct NGS address; but failed to move to operational binaries directory till 2008-09-16 2008-09-12 -- restored ESU & USU clocks in IGU combination due to generally improved performance, starting with 1496_5_12; due to high consistency of ESU & USU clocks with large errors, this causes GFU clocks to be heavily deweighted, so ESU clocks rejected again, starting with 1496_5_18 2008-09-26 -- no ESOC solutions over weekend due to computer work starting this P.M. till 2008-09-29 mid-day 2008-10-17 -- reported to Christine Hackman ongoing discontinuities in USU clock predictions which adversely affect the IGU combination, in particular for 1500_3_18 thru 1500_4_18; will remove USU again from IGU clocks, at least for MX events, if problem is not resolved soon 2008-10-20 -- added link to ACC website to plots maintained by Jan Dousa showing radial, cross-, & along-track orbit differences of the real-time IGU products compared to IGR for each individual SV 2008-10-22 -- again exclude USU clocks in IGU combination due to upcoming MX event scheduled for PRN29 on 2008-10-24, starting with 1502_3_18; meanwhile Christine Hackman reports change of USU clock procedures to omit predictions for SVs with clk breaks; will verify change in upcoming MX events & consider restoration afterwards 2008-10-31 -- major changes of IGU page at ACC website to add new & modify many existing plots; thru 2008-11-05 2008-11-05 -- modified soft/acc_soft/graphics/sum_orb_clk_files.pl to get 06 & 18 HH for IGU summary stats, in addition to 00 & 12 HH; files used for IGU web plots; routine was apparently never updated when frequency of IGU updates was increased starting wk 1267 2008-11-06 -- alert sent to SIO about persistent rejection of SIU solutions from IGU combination due to large Z rotations (since 1504_2_12 on 2008-11-04); alert also sent to ESOC about degraded quality (poor WRMS) of recent ESU orbit solutions for IGU combination; Tim Springer believes problem is mostly due to currently poor quality of IERS Bull A predictions for UT1, which claim was forwarded to the IERS; Tim Springer provided supporting test results to USNO on 2008-11-10 2008-11-09 -- restored USU clocks in IGU combo due to frequent exclusion of GFU solutions caused by large RZ rotations, starting with 1505_0_18 2008-11-10 -- alert sent to GFZ about persistent rejection of GFU solutions from IGU combo due to large Z rotations (since 1504_6_18 on 2008-11-08 all but one submission has been rejected up thru 1505_1_06); problem fixed (mostly) starting 1505_1_12 2008-11-12 -- change remote_daily_autocmb to put $sms_command into background to avoid possible delays due to mail server hangups starting 1505_3_18 2008-11-14 -- again exclude USU clocks in IGU combo due to recent poor quality of PRN05 predictions & failure to detect break in PRN27 clock today, starting with 1505_5_18 2008-11-18 -- alerts sent to GFZ, CODE, & SIO about persistent rejections recently of IGU submissions due to large Z rotations, esp in 1506_1_12, 1506_1_18, 1506_2_00, & 1506_2_06; GFZ & CODE report problems with recent mods to better handle UT1 predictions & are working to fix them 2008-11-19 -- modify remote_daily_autocmb to add new routine to remove clock predictions (replace_sp3_clks.pl) for IGU ACs specified by new exclude_clk_pred option in ultra/inp/initopt_P.tab_ultra; use this feature to exclude IGU clock predictions for ESU & USU while restoring their observed clocks to the combo, starting with 1506_3_12 2008-11-21 -- restored USU clock predictions to IGU combo due to planned outage of EMU solutions on 2008-11-22 (1506_6_18 & 1507_0_00) caused by work on bldg power, starting with 1506_5_18 2008-11-21 -- NOAA/NOS mail server down from ~22:00 till ~06:00 on 2008-11-22 2008-11-22 -- KASI Data Center unreachable from ~10:00 (IGU 1507_0_00) for >21 hr; ftp connection restored by ~20:00 on 2008-11-23 2008-11-25 -- again exclude USU clock predictions in IGU combo after weekend disruption of EMU solutions, starting with 1507_2_18 2008-11-29 -- KASI Data Center unreachable from ~16:00 (IGU 1507_6_18) for >5 hr; ftp connection restored by ~22:00 on 2008-11-29 2008-12-04 -- KASI Data Center unreachable from ~22:00 (IGU 1508_5_00) for >3 d 21 hr; ftp connection restored by ~19:00 on 2008-12-08 2008-12-11 -- notified NRCan of occasional missing clocks in EMU submissions lately; apparently related to ftp problem in their perl deposit script, which will be investigated 2008-12-15 -- ESOC change to updated Napeos version 3.2, mainly to deal with upcoming leap second and to improve handling of EOP parameters & predictions; esp effective for IGU predictions 2008-12-21 -- restored USU clock predictions to IGU combo due to recently improved performance, starting 1511_0_18 2008-12-22 -- restore ESU to IGU LOD plots at ACC website 2009-01-01 -- alert sent to all ACs about year-end & leap-second problems in some AC IGU/IGR solutions, as well as missing solutions from other ACs 2009-01-02 -- ACC report on the status of IGS Ultra products sent to IGS Mail 2009-01-05 -- fixed bug in replace_sp3_clks.pl that caused clock predictions to no longer be excluded for specified ACs in IGU combo; effective starting 1513_1_12 2009-01-05 -- IGN Data Center unreachable from ~16:00 (IGU 1513_1_18) for >11 hr; ftp connection restored by ~04:00 on 2009-01-06 2009-01-06 -- fine-tune some IGU plots at ACC website 2009-01-27 -- considering recent performance improvement, restore ESU clock predictions in IGU combo, starting with 1516_2_18 2009-01-27 -- WRMS for IGU combined 1516_2_12 orbits were 172 mm due to consistent AC problems with PRN16E, whose WRMS was 1046 mm; earlier & later products were OK; IGS Mail #5890 sent on 2009-01-30 to document situation for users 2009-01-28 -- RMS for IGU combined 1516_3_18 near real-time observed clocks was unreasonable apparently due to problem with GFU submission, which did not contain clock predictions, only observations; consequently GFU clocks removed from IGU combo, starting with 1516_4_00 2009-01-29 -- IGU combo 1516_4_18 delivered 20 min late due to problem with ACC server account 2009-02-02 -- modest improvements in USU solution implemented starting with IGU 1517_1_18 2009-02-04 -- IGN Data Center unreachable from ~16:00 (IGU 1517_3_18) for >9 hr; ftp connection restored by ~04:00 on 2009-02-05 2009-02-07 -- NOAA/NOS mail server failure starting by ~16:00 (IGU 1517_6_18) & continuing till shortly before ~10:00 on 2009-02-08; service restored for IGR 1518_0_12 & IGR 1517_6 then failed again by ~16:00 (IGU 1518_0_18) but restored by ~22:00 (IGU 1518_1_00); all summary report mailings failed during affected periods but product deliveries to Data Centers were not affected 2009-02-08 -- CDDIS Data Center unreachable from ~04:00 (IGU 1518_0_06) for >7 hr; ftp connection restored by ~12:00 on 2009-02-08; caused problems with getting RINEX files for PPN & PPP solutions as well as for rintosp3 processing of BRDC nav file 2009-02-08 -- considering recently missing EMU clock predictions & apparent improved reliability, restored GFU clock predictions in IGU combo, starting with 1518_0_18 2009-02-12 -- again exclude ESU clock predictions in IGU combo in the event of problems after maneuver of PRN19 during 2009-02-11 22:34 to 2009-02-12 05:03 & in anticipation of upcoming MX event scheduled on 2009-02-17 for PRN05, starting with 1518_4_18 2009-02-12 -- server mail connection problems resulting in loss of routine combination reports & error/status messages from ~15:00 till ~14:00 on 2009-02-13 (but no effect on product deliveries) 2009-02-12 -- CDDIS Data Center unreachable from ~22:00 (IGU 1518_5_00) for >8.5 hr; ftp connection restored by ~06:30 on 2009-02-13 2009-02-14 -- WRMS for IGU combined 1518_6_06 & _12 orbits were 50 & 42 mm due to consistent AC problems with PRN09 (no eclipse), whose WRMS was 232 & 363 mm, respectively; earlier & later products were OK as well as near RT IGU observed 2009-02-24 -- option to exclude specified PRN clocks from IGU combo when NANU MX events announced tested in IGU 1520_2_12 without success causing 9-min delay in delivery of products 2009-03-04 -- alerted GFZ to frequent rejection of their Ultra solutions due to large Z rotations -- 12 out of most recent 25 combos 2009-03-13 -- IGN Data Center unreachable from ~05:00 (IGU 1522_5_06) for >4.5 hr; ftp connection restored by ~09:35 on 2009-03-13; reported matter to them & told that occasional problems started after a system upgrade a few weeks earlier, but immediate problem solved after ftp server was rebooted 2009-03-26 -- IGN Data Center unreachable from ~11:00 (IGU 1524_4_12) for >3 d, 21 hr; issue reported to them & told of problems with their firewall; ftp connection restored by ~08:00 on 2009-03-30 2009-03-30 -- IGN Data Center unreachable from ~11:00 (IGU 1525_1_12) for ~1 d, 01 hr; issue reported to them & told of continuing problems with their firewall; ftp connection restored by ~12:00 on 2009-03-31 2009-04-13 -- IGN Data Center unreachable from ~11:00 (IGU 1527_1_12) for >5 hr; ftp connection restored by ~17:00 2009-05-12 -- alerted GOP to bad orbit predictions for PRN05 for some weeks 2009-05-13 -- asked CODE group to stop using maneuver flags with SP3 orbit predictions for SVs that have just finished DV events; this causes the IGU combination to include their orbit rather than filter it out, as for all other ACs; caused 3.14 m residual for PRN27 orbit prediction in IGU 1531_2_18 though with very bad accuracy code 2009-05-29 -- alert sent to IGU ACs about some consistent & widespread problems with the orbits of a large number of SVs, mostly from GFU, SIU, & USU 2009-06-01 -- SIU solutions rejected from IGU combinations temporarily due to very poor & erratic performance, starting IGU 1534_1_12 2009-06-01 -- WRMS for IGU combined 1534_1_00 orbits were 134 mm overall (wrt IGR) due to apparent problems with PRN05 (no eclipse) for all 4 contributing ACs; WRMS was 1251 mm for PRN05, which had been set unusable since 2009-03-26; probably at least partially due to bad IGR quality of PRN05 orbit, which came only from NGS; earlier & later products were OK 2009-06-02 -- transfer of ACC plots to website was disrupted till 2009-06-18 due to failure of gateway computer & need to reset ssh keys but during annual leave period 2009-06-08 -- SVN35/PRN05 signal transmissions ended; previously set unusable till further notice on 2009-03-26 2009-06-22 -- SOPAC Data Center unreachable from ~11:00 (IGU 1537_1_12) for >1d 5 hr apparently due to a high-level network router problem; ftp connection restored by ~18:00 on 2009-06-23 2009-06-26 -- alert sent to ACs about setting of PRN25 unusable till further notice, probably due to degrading clock performance; IGU clock ACs asked to temporarily stop reporting clock predictions for PRN25 2009-06-29 -- KASI Data Center unreachable from ~05:00 (IGU 1538_1_06) for >17 hr; ftp connection restored by midnight 2009-06-29 -- IGS Station Mail mesage sent to inform station operators that P2 observations are required over C2 obs whenever there is any inability to report both 2009-06-30 -- modify IGU combination procedures to include a PRN list as well as ACs; use this feature to exclude IGU clock predictions for specified SVs with unstable clocks for all ACs; tested starting with 1538_2_12 2009-07-01 -- begin using new exclude_clk_pred option to exclude IGU clock predictions for: * all 7 IIA SVs with Cs clocks -- PRNs 3,8,9,10,24,27,30 (usually all highly unstable) * all SVs set UNUSUFN -- presently PRNs 1,25 * any SV during MX events -- presently none; at the same time restore ESU clock predictions to combination, starting with IGU 1538_3_12; IGS Mail 5965 sent 2009-07-01 -- after PRN25 set healthy again, it was removed from IGU clock prediction exclusion list, starting with IGU 1538_4_00 2009-07-02 -- added text to IGU combination summary reports under Remark 1 to explain choice of reference AC clock solution used for time scale alignment & the new clock prediction exclusions implemented 2009-07-01, starting with IGU 1538_4_18 2009-07-03 -- 2SOPS restored antenna offset for new SVN49/PRN01 to nominal (from huge antenna offset of ~152 m set on 2009-05-01 to compensate for group delay dispersive anomaly in L1 code signal beam) 2009-07-07 -- added BRD clocks to IGU clock prediction performance plots (compared to IGR) at ACC website 2009-07-08 -- updated IGS products table (file: components/prods.html) sent to CB to post at IGS website 2009-07-08 -- GFU clocks (observed & predicted) temporarily excluded from IGU combination due to poor & unreliable quality, starting with IGU 1539_3_18 2009-07-09 -- alert sent to USNO about poor solutions submitted for IGU 1539_4_00 & IGR 1539_3, which were both totally rejected 2009-07-14 -- IGU clock prediction exclusion policy implemented on 2009-07-01 modified to restore predictions for the IIA satellites with Cs clocks (presently PRNs 3,8,9,10,24,27,30); policy continues to exclude SVs set UNUSUFN (presently PRN 1) & MX (presently none); starting with IGU 1540_2_12; IGS Mail 5969 sent 2009-07-21 -- updated IGS product table posted at CB website 2009-07-22 -- IGN Data Center unreachable from ~05:00 (IGU 1541_3_06) for >5 hr; ftp connection restored by ~12:00 2009-07-22 -- alert sent to USNO about rapidly degrading IGU solution quality 2009-07-23 -- USU orbits rejected from IGU combinations temporarily due to very poor & erratic performance, starting IGU 1541_4_18 2009-07-23 -- GFU clocks (observed & predicted) restored to IGU combination, starting IGU 1541_4_18 2009-07-24 -- due to continuing poor quality GFU clocks (observed & predicted) excluded again from IGU combination, starting IGU 1541_5_12; note that previous exclusion of USU orbits does not affect continued inclusion of their clocks, which are of good quality 2009-07-25 -- IGU clock predictions excluded for PRN18 after being set USUFN, starting IGU 1541_6_12 2009-07-29 -- PRN18 restored to IGU clock predictions after being set healthy again, starting IGU 1542_3_12 2009-08-10 -- IGU predicted orbit for PRN09 (no eclipse) in 1544_1_12 had WRMS error of 368 mm during 12 - 18 UTC apparently due to some unannounced events on the SV; caused overall WRMS error for 6-hr orbit predictions to be 122 mm; PRN09 was OK earlier & was automatically excluded in later IGU combinations due to poor AC agreement; note that a similar period occurred for PRN09 about 6 months earlier (1518_6_06 thru 1519_0_12) & 12 months earlier (1494_2_18 thru 1494_4_06) 2009-08-17 -- ESOC reported that problems with ESA solutions over weekend, esp for IGU, were caused by local IT troubles, which should now be resolved 2009-08-18 -- alert sent to GFZ & GOP about recent degradation in their IGU solution quality 2009-08-19 -- IGU clock predictions excluded for PRN25 after being set UNUSUFN, starting IGU 1545_3_12; ACs alerted & asked to drop PRN25 orbit predictions 2009-08-19 -- ESOC reported renewed IT problems preventing ESA solutions from being submitted, starting IGU 1545_3_12 2009-08-20 -- SVN25/PRN25 set UNUSUFN & decommissioned at 15:14 UTC 2009-08-20 -- GFU orbits rejected from IGU combinations temporarily due to increasingly poor performance, starting IGU 1545_5_00; GFU clocks (observed & predicted) already excluded due to poor quality since 2009-07-24 2009-08-22 -- GFU orbits restored to IGU combinations since performance has improved, starting IGU 1545_6_12; GFU clocks (observed & predicted) continue to be excluded since 2009-07-24 but most recent performance has improved 2009-08-22 -- GOU orbits rejected from IGU combinations temporarily due to increasingly poor performance, starting IGU 1545_6_12 2009-08-26 -- GOU orbits restored to IGU combinations after some problems fixed by Jan Dousa, starting IGU 1546_3_12 2009-08-27 -- SVN50/PRN05 (Block IIR-21/M8) satellite launched on 2009-08-17 set usable at 14:40 UTC 2009-09-06 -- IGU clock predictions excluded for PRN05 after being set UNUSUFN, starting IGU 1548_0_12 2009-09-07 -- IGU clock predictions restored for PRN05 after being set healthy again, starting IGU 1548_1_00 2009-09-09 -- alert sent to CODE group about large Z rotations in their IGU submissions causing rejections, starting 1548_3_00 & ending 1548_3_18 2009-09-10 -- alert sent to GFZ group about numerous missing SV orbits in their recent IGU submissions 2009-09-12 -- IGU clock predictions excluded for PRN24 after being set UNUSUFN, starting IGU 1548_6_18 2009-09-12 -- IGU clock predictions restored for PRN25 after being set healthy again, starting IGU 1549_0_00 2009-09-25 -- CDDIS Data Center unreachable from ~23:00 (IGU 1550_6_00) for >1 d 19 hr; ftp connection restored by ~19:30 on 2009-09-27; lots of problems caused by this outage, for ACs and ACC combos, for many various different reasons; PPN part of IGR summaries missing for 1550_5 & 1550_6 due to lack of station data from CDDIS; PPP reports for same days distributed late due to lack of station data from CDDIS 2009-09-26 -- GFU clocks temporarily restored to IGU combos due to lack of redundant clock ACs, starting IGU 1551_0_00; somewhat risky considering that they sometimes experience large resets 2009-09-26 -- IGU combo 1551_0_00 submitted ~30 min late due to failure of the automated process, apparently due to a missing subdirectory which seems to be caused by CDDIS not being operational; missing subdirectory created by hand, latest IGU ACs solution files retrieved manually, & combo restarted 2009-09-27 -- due to AC problems, most likely related to CDDIS being down, only two solutions used in IGU combos 1551_0_06 (EMU & GFU) & 1551_0_18 (COU & EMU) 2009-09-27 -- alert sent to NRCan group about somewhat poorer IGU orbit performance, esp for median residuals, than usual & frequent missing clocks 2009-09-28 -- GFU clocks again excluded from IGU combos due to sometimes large resets or bad alignment to GPS time, as seen in IGU 1551_1_06 for instance, starting 1551_1_12 2009-09-28 -- PRN26 clock predictions excluded from IGU combos due to scheduled MX event, starting 1551_1_12 2009-09-30 -- IGU clock predictions restored for PRN26 after end of MX event, starting IGU 1551_4_00 2009-10-11 -- another alert sent to NRCan group about degrading IGU orbit performance, following period of improved performance 2009-10-14 -- PRN08 clock predictions excluded from IGU combos due to scheduled UNUSUFN outage, starting 1553_4_00 2009-10-15 -- IGU predicted orbit for PRN08 (no eclipse) in 1553_4_12 had WRMS error of 270 mm during 12 - 18 UTC after being set UNUSUFN earlier, which caused overall WRMS error for 6-hr orbit predictions to be 58 mm; residuals increased to 1080 mm for PRN08 & 93 mm overall WRMS in IGU 1553_4_18; there were not any similar periods about 6 months or 1 year earlier for PRN08 2009-10-21 -- alert sent to GFZ group about markedly poorer IGU orbit performance than usual since about IGU 1553_6_00, though their median orbit residual is not bad enough to warrant exclusion at this time 2009-10-24 -- PRN30 clock predictions excluded from IGU combos due to series of scheduled MX events, starting 1554_6_18 2009-10-25 -- after extremely bad IGU orbits received from GOU for 1555_0_18 they were excluded temporarily from future combos until improvements seen, starting 1555_1_00; Jan Dousa reported work underway to repair problems on 2009-10-26 2009-10-26 -- PRN25 clock predictions excluded from IGU combos due to scheduled UNUSUFN outage, starting 1555_1_12; subsequently the SV stopped transmitting standard L-band signals 2009-10-27 -- GOU orbits restored to IGU combinations after problems fixed by Jan Dousa, starting IGU 1555_2_18 2009-10-29 -- PRN08 clock predictions restored to IGU combos following end of UNUSUFN outage, starting 1555_4_12 2009-10-30 -- follow-up alert sent to GFZ group about increasingly poor IGU orbit performance; their RMS & WRMS residuals have continued to increase but now the median orbit residuals are also quite poor; it might be necessary to start excluding their contributions soon 2009-10-30 -- PRN18 clock predictions excluded from IGU combos due to scheduled MX events, starting 1555_5_12 2009-11-02 -- GFU orbits rejected from IGU combinations temporarily due to increasingly poor performance, starting IGU 1556_1_12 2009-11-02 -- PRN25 L-band signal transmissions resumed, but still set UNUSUFN 2009-11-03 -- PRN18 clock predictions restored to IGU combos following end of MX outages, starting 1556_2_12 2009-11-03 -- IGU predicted orbit for PRN32 (no eclipse) in 1556_2_00 had WRMS error of 270 mm during 00 - 06 UTC, which caused overall WRMS error for 6-hr orbit predictions to be 62 mm; residuals decreased in IGU 1556_2_06 to 200 mm for PRN32 but PRN04 (in eclipse) had WRMS of 227 mm giving an overall WRMS of 42mm; normal performance levels were restored in IGU 1556_2_12; there was a similar period nearly 1 year earlier for PRN32 in IGUs 1507_3_12 & _18 2009-11-05 -- PRN30 clock predictions restored to IGU combos following end of MX outages, starting 1556_4_12 2009-11-05 -- PRN08 clock predictions excluded from IGU combos due to scheduled UNUSUFN outage, starting 1556_4_18 2009-11-11 -- GFU orbits restored to IGU combinations following large improvements to usual quality, starting IGU 1557_3_12; GFU clocks (observed & predicted) continue to be excluded due to erratic quality since 2009-07-24 2009-11-13 -- PRN25 clock predictions restored to IGU combos following outages, starting 1557_5_18 2009-11-23 -- PRN08 clock predictions restored to IGU combos following end of UNUSUFN period, starting 1559_1_12 2009-12-03 -- CODE AC down due to computer security problems at AIUB, starting with IGU 1560_5_00; normal operations restored 2009-12-09, starting with IGR 1561_2 & IGU 1561_3_12 2009-12-05 -- PRN05 clock predictions excluded from IGU combos due to scheduled MX outage, starting 1560_6_12 2009-12-05 -- ESOC AC down also due to unreported problem, starting with IGU 1561_0_00; normal operations restored 2009-12-07 with IGR 1561_0 & IGU 1561_1_18; during this period both CODE & ESOC were missing from all IGU & IGR combos but IGU product quality not significantly affected 2009-12-08 -- GFZ AC also down briefly while CODE was still out, due to unreported problem; missed IGUs 1561_2_18, _3_00, & _3_06 with minimal impact 2009-12-09 -- PRN05 clock predictions restored to IGU combos following end of MX outage, starting 1561_3_12 2009-12-09 -- partial CODE AC operations restored restored, starting with IGR 1561_2 & IGU 1561_3_12 2009-12-16 -- PRN25 clock predictions excluded from IGU combos due to planned UNUSUFN outage, starting 1562_3_18 2009-12-16 -- unscheduled, site-wide internet outage sometime between 16:00 & 22:00 EST caused generation & delivery of IGUs 1562_4_00 & _4_06 to be delayed, by 10h 40m & 4h 45m, respectively; IGS Mail 6039 sent to notify users 2009-12-19 -- full operations restored at CODE AC after shutdown on 2009-12-03 due to computer security problems at AIUB 2009-12-22 -- CDDIS Data Center down starting ~21:00 EST apparently due to problems with the RAID system; problems resolved ~20:00 EST on 2009-12-23 2009-12-23 -- SOPAC Data Center down starting ~15:00 EST due to unknown problem; issue resolved ~01:00 EST on 2009-12-24 2009-12-28 -- IGU clock predictions restored for PRN24 after being set UNUSUFN since 2009-09-12, starting IGU 1564_1_12 2010-01-07 -- report on status of IGS orbit products sent to IGS Mail (#6053), UNAVCO exploder, & CANSPACE 2010-01-21 -- starting this week ESOC has added some stations to their RF set, esp ALRT, NICO, NKLG, NRIL, OHI2, SCUB, WHIT, which might improve their orbit frame rotations; also moved to a new, faster NAPEOS version 3.4; also starting applying the ocean tidal loading CMC correction in their IGU products; also working on adding GLONASS to their IGR & IGU solutions, which will change the contents of their SP3 files 2010-01-25 -- PRN04 clock predictions excluded from IGU combos due to scheduled MX outage, starting 1568_1_12 2010-01-26 -- PRN07 clock predictions excluded from IGU combos due to scheduled MX outage, starting 1568_2_18 2010-01-30 -- IGU clock predictions restored for PRNs 04 & 07 after scheduled MX outages, starting 1568_6_18 2010-02-01 -- IGU combos 1569_1_00 & _1_06 delayed till ~13:30 UTC due to inability to download AC solution submissions, apparently caused by an overnight network outage of unknown nature 2010-02-02 -- PRN21 clock predictions excluded from IGU combos due to scheduled MX outage, starting 1569_2_12 2010-02-07 -- IGU clock predictions restored for PRN21 after scheduled MX outage & excluded for PRN31 due to scheduled MX outage, starting 1570_1_06 2010-02-08 -- SVN25/PRN25, which was officially decommissioned on 2009-12-18, finally stopped broadcasting L1/L2 signals 2010-02-11 -- SVN35 reactivated (formerly PRN05) & assigned to PRN25, which was previously SVN25 2010-02-12 -- IGU clock predictions restored for PRN31 after scheduled MX outage, starting 1570_5_18 2010-02-13 -- GFU orbits rejected from IGU combinations temporarily due to increasingly poor performance, starting IGU 1570_6_18; GFU clocks (observed & predicted) already excluded due to sometimes erratic quality since 2009-07-24 2010-02-14 -- PRN02 clock predictions excluded from IGU combos due to scheduled MX outage, starting 1571_0_12 2010-02-15 -- PRN20 clock predictions excluded from IGU combos due to scheduled UNUSUFN outage, starting 1571_1_12 2010-02-16 -- PRN23 clock predictions excluded from IGU combos due to scheduled MX outage, starting 1571_2_12 2010-02-16 -- IGU clock predictions restored for PRN23 following cancellation of scheduled MX outage, starting 1571_2_18 2010-02-18 -- IGU clock predictions restored for PRN02 after scheduled MX outage, starting 1571_4_18 2010-02-20 -- PRN06 clock predictions excluded from IGU combos due to scheduled MX outage, starting 1571_6_12 2010-02-22 -- IGU clock predictions restored for PRN20 after scheduled UNUSUFN outage, starting 1572_1_12 2010-02-22 -- PRN16 clock predictions excluded from IGU combos due to scheduled MX outage, starting 1572_1_12 2010-02-22 -- PRN30 clock predictions excluded from IGU combos due to scheduled UNUSUFN outage, starting 1572_2_00 2010-02-24 -- IGU clock predictions restored for PRN16 after cancelled MX outage, starting 1572_3_12 2010-02-25 -- IGU clock predictions restored for PRN06 after scheduled MX outage & for PRN30 after scheduled UNUSUFN outage, starting 1572_4_18 2010-02-27 -- alert sent to ACs about large station displacements in Chile & southern South America due to magnitude 8.8 offshort earthquake earlier today; according to a solution by Kristine Larson, CONZ/CONT moved by ~2.5 m in the E-W direction & by ~0.6 m in the N-S direction 2010-02-28 -- PRN27 clock predictions excluded from IGU combos due to scheduled MX outage, starting 1573_0_12 2010-03-01 -- PRN15 clock predictions excluded from IGU combos due to scheduled MX outage, starting 1573_1_12 2010-03-04 -- IGU clock predictions restored for PRN27 after scheduled MX outage, starting 1573_4_12 2010-03-05 -- IGU clock predictions restored for PRN15 after scheduled MX outage, starting 1573_5_18 2010-03-15 -- 1st use of GPS+GLO products from ESA in IGU combinations, starting with 1575_1_12; details given by them in IGS Mail 6107 2010-03-16 -- due to an unannounced URL change for the Air Force website source of NANUs, an announced maneuver for PRN32 today was not detected and properly deleted in the most recent IGUs; problem corrected starting IGU 1575_3_00; IGU vs IGR comparison for 1575_2_18 has very large orbit residuals due to this problem 2010-03-23 -- PRN22 clock predictions excluded from IGU combos due to scheduled MX outage, starting 1576_2_18 2010-03-24 -- PRN09 clock predictions excluded from IGU combos due to extension of prior DV outage, starting 1576_3_12 2010-03-25 -- IGU clock predictions restored for PRN22 after scheduled MX outage was cancelled, starting 1576_4_12 2010-03-28 -- PRN12 clock predictions excluded from IGU combos due to scheduled MX outage, starting 1577_0_12 2010-03-29 -- PRN22 clock predictions excluded from IGU combos due to scheduled MX outage, starting 1577_1_12 2010-03-31 -- IGU clock predictions restored for PRN12 after scheduled MX outage, starting 1577_3_12 2010-03-31 -- PRN19 clock predictions excluded from IGU combos due to scheduled MX outage, starting 1577_3_12 2010-04-02 -- IGU clock predictions restored for PRN22 & PRN19 after scheduled MX outages, starting 1577_5_18 2010-04-04 -- PRN28 clock predictions excluded from IGU combos due to scheduled MX outage, starting 1578_0_18 2010-04-05 -- PRN29 clock predictions excluded from IGU combos due to scheduled MX outage, starting 1578_2_00 2010-04-07 -- IGU clock combo for 1578_3_12 degraded due to problems with the ESU solution 2010-04-08 -- IGU clock predictions restored for PRN28 after scheduled MX outage, starting 1578_4_18 2010-04-10 -- IGU clock predictions restored for PRN29 after scheduled MX outage, starting 1578_6_12 2010-04-12 -- PRN18 & PRN14 clock predictions excluded from IGU combos due to scheduled MX outages, starting 1579_1_12 2010-04-13 -- IGU clock predictions restored for PRN18 due to rescheduling of MX outage, starting 1579_2_12 2010-04-14 -- IGU clock predictions restored for PRN09 after extended DV outage, starting 1579_3_18 2010-04-16 -- IGU clock predictions restored for PRN14 after scheduled MX outage, starting 1579_5_18 2010-04-19 -- PRN13 clock predictions excluded from IGU combos due to scheduled MX outage, starting 1580_1_18 2010-04-20 -- PRN18 clock predictions excluded from IGU combos due to scheduled MX outage, starting 1580_2_12 2010-04-21 -- IGU clock predictions restored for PRN13 after scheduled MX outage & PRN23 clock predictions excluded from IGU combos due to scheduled MX outage, starting 1580_3_18 2010-04-22 -- IGU clock predictions restored for PRN18 after scheduled MX outage, starting 1580_4_18 2010-04-24 -- IGU clock predictions restored for PRN23 after scheduled MX outage, starting 1581_0_00 2010-04-25 -- PRN17 clock predictions excluded from IGU combos due to scheduled MX outage, starting 1581_0_12 2010-04-28 -- PRN11 clock predictions excluded from IGU combos due to scheduled MX outage, starting 1581_3_12 2010-04-28 -- IGN Data Center unreachable from ~13:00 EDT (IGR 1581_2) for >15 hr; ftp connection restored by ~06:00 on 2010-04-29 2010-04-29 -- IGU 1581_4_12 posted late due to failure of a major internet truck line at ~07:45 EDT; service restored at 14:57 EDT & late IGU & IGR solutions resubmitted; IGS Mails 6140 & 6141 sent to report status to users 2010-04-29 -- IGU clock predictions restored for PRN17 after scheduled MX outage, starting 1584_4_18 2010-05-02 -- IGU clock predictions restored for PRN11 after scheduled MX outage, starting 1582_0_12 2010-05-02 -- PRN16 clock predictions excluded from IGU combos due to scheduled MX outage, starting 1582_0_12 2010-05-03 -- PRN05 clock predictions excluded from IGU combos due to scheduled MX outage, starting 1582_1_12 2010-05-03 -- IGN Data Center unreachable from ~11:00 EDT (IGU 1582_1_12) for >17 hr; ftp connection restored by ~04:00 on 2010-05-04 2010-05-06 -- IGU clock predictions restored for PRN16 & PRN05 after scheduled MX outages, starting 1582_4_06 2010-05-06 -- PRN09 clock predictions excluded from IGU combos due to scheduled MX outage, starting 1582_4_06 2010-05-08 -- IGU clock predictions restored for PRN09 after scheduled MX outage, starting 1582_6_06 2010-05-12 -- PRN13 clock predictions excluded from IGU combos due to extension of prior DV outage, starting 1583_3_12 2010-05-21 -- PRN32 clock predictions excluded from IGU combos due to scheduled UNUSUFN outage, starting 1584_5_12 2010-05-21 -- PRN16 clock predictions excluded from IGU combos due to scheduled UNUSUFN outage, starting 1584_5_18 2010-05-23 -- IGU clock predictions restored for PRN13 after extended DV outage, starting 1585_0_12 2010-05-28 -- IGN Data Center unreachable from ~05:00 EDT (IGU 1585_5_06) for >2 d 23 hr; ftp connection restored by ~04:00 on 2010-05-31 2010-06-01 -- GFZ starts submitting GPS+GLO combined IGU solution files beginning 1586_2_12 2010-06-07 -- IGU clock predictions restored for PRN16 & PRN32 after scheduled UNUSUFN outages, starting 1587_1_12 2010-06-12 -- ESU LODs excluded from IGU combos due to unphysical LOD predictions, starting 1588_0_00 2010-06-13 -- 152 m antenna offset in SVN49/PRN01 BRDC orbit removed, apparently starting late on 2010-06-12; 2SOPS confirms the change was intentional but cannot say if it will remain indefinitely or what other changes might be made; this is apparently a test to collect more data for the SV in this state to use to decide how to configure the SV permanently 2010-06-15 -- user asked about possible causes of recent poor IGU prediction performance for PRN23 (after leaving a period of lunar eclipsing), which is unknown, & about apparent changes in ESU performance, which is not seen in ACC stats 2010-06-24 -- PRN16 clock predictions excluded from IGU combos due to scheduled UNUSUFN outage, starting 1589_4_18 2010-06-27 -- IGU clock predictions restored for PRN16 after scheduled UNUSUFN outage, starting 1590_0_12 2010-07-13 -- a problem reported by CODE on 2010-06-28 with more recently uploaded solution files not getting used in our processing was resolved by modifying the download perl-ftp script logic to explicitly set the file modification time to match the time from the source data center; the problem appeared after the change in CDDIS server on 2010-06-22, apparently due to a different server directory configuration; the 1st bug fix was incomplete due to later file times for the copies put into processing directories, so a further mod was made on 2010-07-14 to preserve the file times when they are copied 2010-07-19 -- responded to IERS Call for Participation in Pilot Project for Combination of EOP Predictions to inform about IGS Ultra-rapid products 2010-07-21 -- a user asked about larger orbit errors for some SVs in IGU products during 1592_5, _6, and 1593_0; this apparently causes his Kalman filter that estimates RT clocks to fail sometimes; a check indicates that the performance was indeed poor for a larger-than-normal number of IIR & IIR-M SVs but it is not clear why this happened 2010-07-26 -- GFU orbits & ERPs restored to IGU combos due to recent major improvements in quality, starting 1594_2_12; GFU orbits/ERPs had been manually excluded since 2010-02-13; GFU clocks (observed & predicted) continue to be excluded (since 2009-07-24) due to sometimes erratic quality that can corrupt combined values 2010-08-05 -- updated NAPEOS software (ver 3.5) implemented by ESOC, starting with IGU 1595_4_12; main difference is shift from Sun to Linux platforms, which allows 4 times faster run times & larger sets of tracking stations (from 85 to 110); plus, GLONASS code obs now included too & LOD prediction error fixed 2010-08-09 -- ESU LODs restored to IGU LOD combination after verifying that NAPEOS software update (ver 3.5) on 2010-08-05 really does seem to resolve prior problems, starting with IGU 1596_1_12 2010-08-09 -- fixed bug (reported by CODE group) that prevented the number of stations used by COU in IGU combos from being listed correctly in 06 hr (only) reports, starting with IGU 1596_1_12 2010-08-25 -- PRN26 clock predictions excluded from IGU combos due to scheduled MX outage, starting 1598_4_12 2010-08-27 -- 1st IIF SV, SVN62/PRN25, set healthy at 04:10 2010-08-30 -- IGU clock predictions restored for PRN26 after scheduled MX outage, starting 1599_1_18 2010-08-31 -- GFZ changed their IGU clock prediction strategy to try to fix problems with large outliers, starting 1599_2_06 2010-09-09 -- IGU combination 1600_5_00 had only 2 usable ACs (COU & ESU), plus SIU & USU, which are not included; EMU was out due to local problems with new computer 2010-09-10 -- NRCan group reported problems with AOA Benchmark receivers tracking P2 code signals since 2010-09-07, evidently due to the flex-power testing underway till 2010-09-11; problem was not so bad as to cause loss of significant data from 30s RINEX files but was apparently much worse for high-rate RT data systems; problems resolved when flex-power testing ended 2010-09-11 -- IGU orbits (observed & predicted) started getting gradually worse during the 1st half of the day but became very bad in the 2nd half; the WRMS orbit residual for the IGU combined predictions for 1600_6_12 (wrt IGR) reached 48 cm overall during the 1st 6 hr of predictions & was 4.10 m for PRN21; but IGR orbit for PRN21 itself was not necessarily reliable; problems persisted in IGU combo thru 1601_0_00; cause unknown 2010-09-14 -- considering changes made by GFZ on 2010-08-31 to improve their IGU clock prediction strategy, GFU clocks restored to IGU combo starting 1601_2_12 2010-09-15 -- GOP starts submitting GPS+GLO combined IGU solution files beginning 1601_3_12 (together with CODE, ESOC, & GFZ) 2010-09-15 -- PRN30 clock predictions excluded from IGU combos due to immediate UNUSUFN outage, starting 1601_3_18 2010-09-17 -- IGU clock predictions restored for PRN30 after UNUSUFN outage; also PRN25 clock predictions allowed into IGU clock combo for 1st time since launch; both starting 1601_5_12 2010-09-21 -- no PRN25 orbit results in IGU 1602_2_12 combination due to a severe problem with the solution from one AC 2010-09-28 -- PRN25 clock predictions excluded from IGU combos due to scheduled MX outage, starting 1603_2_12 2010-10-01 -- IGU clock predictions restored for PRN25 after scheduled MX outage, starting 1603_5_18 2010-10-19 -- CODE implemented several analysis changes in their Final solutions, starting wk 1604; changes include: replace GMF with VMF1 mapping function & adding 2nd order, 3rd order, & ray bending corrections for iono delay; first two iono changes already applied in CODE IGR & IGU submission since 2010-02-15 with 3rd term added on 2010-09-24 2010-10-31 -- PRN22 clock predictions excluded from IGU combos due to scheduled UNUSUFN outage, starting 1608_0_12 2010-11-03 -- IGU clock predictions restored for PRN22 after scheduled UNUSUFN outage, starting 1608_3_18 2010-11-24 -- ESA found that a problem reported 2010-11-10 with their Rapid ERPs was caused by a bug that ignores all tracking data from stations with names alphabetically later than NOTE; problem fixed starting IGU 1611_3_18 & IGR 1611_3 2010-11-28 -- the IGV GLONASS combination for 1612_0_12 failed due to having only 1 AC contribution (ESA); the corresponding IGU GPS combo had only 2 ACs; CODE had an unexpected system outage over the weekend, which stopped all their products for a while, & GOP ran into a bug that was recently introduced; a combined IGV SP3 file was issued as normal but all the GLONASS SV orbit accuracy codes were 0; several other recent IGVs were near-failures with only 2 ACs; ACs were notified on 2010-11-29 2010-11-29 -- to reduce problems with large AC outliers in IGU/IGV combos (as seen recently with GFU in 1611_5_12, COU in 1611_6_12, & GFU in 1612_0_18) the rejection criteria were tightened considerably; change implemented in IGU/IGV combos starting 1612_1_12 2010-11-30 -- Andre Hauschild (DLR) announced the availability of new GLONASS RT products, in addition to GPS+GIOVE; see IGS Mail 6302; he uses the new IGV test GLO orbits as the basis for this new service that adds GLO clocks 2010-12-15 -- technical report on SVN47/PRN22 signal anomaly issued by Andre Hauschild (DLR); see IGS Mail 6319 2010-12-21 -- PRN04 clock predictions excluded from IGU combos due to rescheduled MX outage, starting 1615_2_12 2010-12-22 -- IGU clock predictions restored for PRN04 after cancellation of rescheduled MX outage, starting 1615_3_12 2010-12-23 -- warning sent to IGS Mail (#6321) about problematic orbit for PRN25/SVN62 that began ~06:00 UTC on 2010-12-22 (IGU 1615_3_06) as PRN25 neared the middle of its eclipse season; compared to IGR orbits, 6-hr IGU prediction residuals were: 18 cm in 1615_3_06, 79 cm in 1615_3_12, 156 cm in 1615_4_00, & rejected from other IGUs until PRN25 orbit began to improve with 16 cm residual at ~18:00 on 2010-12-24 (IGU 1615_5_18); approached nearly normal state by ~12:00 on 2010-12-25 (IGU 1615_6_12) 2011-01-03 -- PRN04 clock predictions excluded from IGU combos due to scheduled MX outage, starting 1617_1_12 2011-01-05 -- the IGV GLONASS combination for 1617_3_18 failed apparently due to very problematic solutions from ESA & GOP, though the exact cause was unclear; a new solution was resubmitted 1h 36m later using only the CODE & GFZ inputs; contributions from ESA & GOP continued to be fully excluded overnite (for 1617_4_00 & _06) to allow ACs time to check their processings; ACs were notified via IGS-ACS; Jan Dousa reported that the GOP solution suffered from not taking proper account of a change in the satellite assignment for R17 (related to the igs05_1617.atx update) 2011-01-06 -- restored ESA & GOP to IGV GLONASS combination but only for comparison initially, starting IGV 1617_4_12 2011-01-06 -- fully restored ESA & GOP to IGV GLONASS combination after test in 1617_4_12 showed prior problems seem to be resolved, starting IGV 1617_4_18 2011-01-07 -- IGU clock predictions restored for PRN04 after scheduled MX outage, starting 1617_5_18 2011-01-10 -- PRN06 clock predictions excluded from IGU combos due to scheduled MX outage, starting 1618_1_12 2011-01-13 -- IGU clock predictions restored for PRN06 after scheduled MX outage, starting 1618_4_12 2011-01-15 -- IGU & IGV 1618_6_12 rerun & resubmitted (about 30 & 45 min after regular runs) due to very poor GFU clocks; ACs notified & GFU clocks continued to be excluded 2011-01-20 -- IGU orbits (observed & predicted) became worse than normal during the 1st 6 hr of the day due to anomalous problems with PRN03, which had a WRMS orbit residual (wrt IGR) of 55.9 cm; this caused the overall WRMS for the 1st 6 hr of 1619_4_00 to be 7.3 cm; while the IGU orbit quality for PRN03 continued to be poor during the rest of the day, its impact on the overall IGU WRMS was minor later in the day; the AC solutions for PRN03 in IGR 1619_4 were also poorer than normal, indicating some sort of unmodeled motion of PRN03 (not in eclipse) early in the day 2011-01-24 -- PRN25 clock predictions excluded from IGU combos due to scheduled MX outage, starting 1620_1_12 2011-01-24 -- IGU clock predictions restored for PRN25 after cancellation of scheduled MX outage, starting 1620_1_18 2011-01-25 -- added new ACC website plots to show orbit Median differences for ACs in IGU GLONASS (IGV) combinations, in addition to prior WRMS plots, & updated some new publications 2011-01-28 -- PRN10 clock predictions excluded from IGU combos due to short- notice UNUSUFN outage, starting 1620_5_18 2011-01-30 -- IGU clock predictions restored for PRN10 after UNUSUFN outage, starting 1621_0_18 2011-01-31 -- GFZ clocks no good in IGU & IGV 1621_1_12, but they were not included, which justifies continuing to exclude them; the problem was apparently not during the 1st 6 hr of predictions however 2011-02-01 -- IERS EOP services adopt new 08C04 series consistent with ITRF2008 2011-02-01 -- GFZ clocks again no good in IGU & IGV 1621_2_06, but still not included; the problem was apparently caused by their estimates for PRN23 but again not during the 1st 6 hr of predictions 2011-02-04 -- GFZ GLO clocks apparently again no good in IGV 1621_5_18, but based only on comparison with ESA GLO clocks 2011-02-08 -- updated ACC website to add chronology of IGS reference frame usage 2011-02-10 -- only 2 ACs (ESA & GFZ) available for IGVs 1622_5_00, _06 & _12 with 3 ACs for matching IGUs (with EMR also) 2011-02-11 -- GFZ GPS & GLO clocks again no good in IGU/IGV 1622_5_18, but still not included in combinations 2011-02-12 -- only 2 ACs (ESA & GFZ) available for IGV 1622_6_06, with 3 ACs for matching IGU (with EMR also) 2011-02-13 -- only 2 ACs (ESA & GFZ) available for IGV 1623_0_06, with 3 ACs for matching IGU (with EMR also) 2011-02-14 -- massive site-wide power failure at ~10:00 EST caused delay in publication of combination products IGU/IGV 1623_1_12 & IGR 1623_0; power restored at ~12:08 EST for a total delay of ~2 hr for the IGU/IGV & ~8 min for the IGR 2011-02-15 -- Georg Weber (BKG) announced start of RT stream from WTZZ carrying GPS+GLO+GIOVE obs & nav data 2011-02-20 -- GFZ GPS & GLO clocks again no good in IGU/IGV 1624_0_12, but still not included in combinations 2011-02-21 -- PRN06 clock predictions excluded from IGU combos due to scheduled MX outage, starting 1624_2_00 2011-02-25 -- IGU clock predictions restored for PRN06 after scheduled MX outage, starting 1624_5_18 2011-03-05 -- IGN Data Center unreachable from ~22:00 EDT (IGU 1626_0_00) for >11 hr; ftp connection restored by ~09:00 on 2011-03-06 2011-03-07 -- IGS Mails 6354, 6355, & 6356 sent to inform user community of IGS switch to IGS08 + igs08.atx planned for 2011-04-17 = GPS wk 1632 with detailed information included on effects of these updates & practical advice on how to cope with the changes; ACC website was updated with references to these messages 2011-03-08 -- modified ACC website plots for missing IGU SV orbits to allow up to 32 SVs to be visible 2011-03-13 -- PRN27 clock predictions excluded from IGU combos due to scheduled MX outage, starting 1627_1_00 2011-03-15 -- PRN25 clock predictions excluded from IGU combos due to scheduled MX outage, starting 1627_2_18 2011-03-16 -- IGU clock predictions restored for PRN27 after scheduled MX outage, starting 1627_3_18 2011-03-17 -- only 2 ACs (COD & GFZ) available for IGV 1627_4_18, with 3 ACs for matching IGU (with EMR also) 2011-03-17 -- IGU orbits (observed & predicted) became much worse than normal during 1627_4_00 due to anomalous problems with PRN25, which had a WRMS orbit residual (wrt IGR) of 420.0 cm; this caused the overall WRMS for the 1st 6 hr of 1627_4_00 to be 15.4 cm; later IGU updates for that day excluded PRN25 from the combination due to too few & inconsistent AC solutions; this event occurred just following the end of a scheduled MX NANU event that had been forecast to run from 2011-03-16 at 18:30 till 18:30 the next day, but which was cut short & actually ran from 19:05 till 23:38; note that the IGR combination 1627_3 for 2011-03-16 had highly discordant AC solutions for PRN25 of which 2 out of 6 submissions were rejected but the IGR 1627_4 combination was normal; these factors indicate that some sort of changes were made onboard the SV (not in eclipse) that caused unpredicted motion even though MX events normally do not affect the SV dynamics; note also that the normal 6+12 hr periodic variations of the PRN25 clock were greatly amplified during the MX outage, though some portion of this could be an artifact of poorer IGR orbits; a new MX outage is now scheduled for PRN25 on 2011-03-21 at 21:00 2011-03-26 -- GFZ GPS & GLO clocks again no good in IGU/IGVs 1629_0_00, _0_06, _0_12, & _0_18 but still not included in combinations 2011-03-28 -- IGU clock predictions restored for PRN25 after scheduled MX outage, starting 1629_1_18 2011-04-01 -- GOP reported that they found & fixed a bug in their Ultra-rapid production dealing with the handling of stochastic parameter estimates in the predictions, which has recently been causing degraded performance 2011-04-03 -- GFZ GPS & GLO clocks again no good in IGU/IGVs 1630_0_06 but still not included in combinations 2011-04-09 -- only 2 ACs (ESA & GFZ) available for IGVs 1630_6_06 & 1631_0_00, with 3 ACs for matching IGUs (with EMR also) 2011-04-10 -- PRN06 clock predictions excluded from IGU combos due to scheduled MX outage, starting 1631_0_12 2011-04-10 -- PRN26 clock predictions excluded from IGU combos due to after- the-fact UNUSUFN announcement, starting 1631_0_18 2011-04-10 -- only 2 ACs (ESA & GFZ) available for IGV 1631_0_18, with 3 ACs for matching IGU (with EMR also) 2011-04-12 -- only 2 ACs (ESA & GOP) available for IGV 1631_3_00, with 3 ACs for matching IGU (with EMR also) 2011-04-14 -- IGU clock predictions restored for PRN26 following end of UNUSUFN outage, starting 1631_4_18 2011-04-14 -- IGU clock predictions restored for PRN06 after scheduled MX outage, starting 1631_4_18 2011-04-17 -- IGS adopted new IGS08 reference frame & igs08.atx antenna calibrations in all its products (starting 1632_0_00) 2011-04-18 -- GFZ GPS & GLO clocks again no good in IGU/IGVs 1632_1_12 & _18 but still not included in combinations 2011-04-19 -- only 2 ACs (GFZ & GOP) available for IGV 1632_2_06, with 3 ACs for matching IGU (with EMR also) 2011-04-19 -- only 2 ACs (COD & ESA) available for IGV 1632_2_18, with 3 ACs for matching IGU (with EMR also) 2011-05-07 -- PRN10 clock predictions excluded from IGU combos due to UNUSUFN announcement, starting 1634_6_12 2011-05-13 -- IGU clock predictions restored for PRN10 following end of UNUSUFN outage, starting 1635_5_06 2011-05-13 -- PRN30 clock predictions excluded from IGU combos due to UNUSUFN announcement, starting 1635_6_00 2011-07-27 -- PRN06 clock predictions excluded from IGU combos due to scheduled MX outage, starting 1646_3_12 2011-07-30 -- IGU clock predictions restored for PRN06 after scheduled MX outage, starting 1647_0_00 2011-08-02 -- GOU temporarily removed from the IGU combinations due to using only about half of the GPS constellation since 1647_1_12; the earlier GOU submissions since 1647_0_18 were missing; no changes applied to IGV GLONASS combination since recent GOU submissions have been missing; IGU exclusion started with 1647_2_18 but the GOU submission was complete again starting then (though still missing a GLONASS solution until a partial submission for 1647_3_00); GOU GPS submissions were restored to the combination on 2011-08-03 starting with IGU 1647_3_18 2011-08-05 -- SOPAC Data Center unreachable from ~23:00 EDT (IGU 1647_6_00) for >23 hr; ftp connection restored by ~23:00 on 2011-08-06 2011-08-09 -- PRN27 clock predictions excluded from IGU combos due to UNUSUFN announcement, starting 1648_2_18 2011-08-26 -- SOPAC Data Center unreachable from ~17:00 EDT (IGU 1650_5_18) for >19 hr; ftp connection restored by ~12:00 on 2011-08-27 2011-08-27 -- SOPAC Data Center unreachable again from ~13:00 EDT (IGR 1650_5) for >21 hr; ftp connection restored before ~12:00 on 2011-08-28 2011-09-08 -- SOPAC Data Center unreachable again from ~23:00 EDT (IGU 1652_5 _00) for >15 hr due to massive power outage affecting a large part of S. California; ftp connection restored before ~16:00 on 2011-09-09; SIO analysis products disrupted during the same time period 2011-09-11 -- SOPAC Data Center unreachable again from ~11:00 EDT (IGU 1653_0 _12) for >29 hr; ftp connection restored before ~18:00 on 2011-09-12 (but SIO AC continued to function OK during this period) 2011-09-15 -- considering recently poorer orbit Z rotations from CODE in IGUs the rejection.tab criterion for rotations was tightened from 1.0 to 0.5 mas in IGU/IGV combos starting 1653_4_12; the last change in the IGU/IGV rejection.tab file was made on 2010-11-29 but that was for outliers, not rotations; the last decrease in the IGU/IGV rotation threshold was from 5.0 to 1.0 mas on 2008-07-14 2011-09-21 -- only 2 ACs (COD & ESA) available for IGV 1654_3_06, with 3 ACs for matching IGU (with EMR also) 2011-09-30 -- PRN08 clock predictions excluded from IGU combos due to UNUSUFN announcement, starting 1655_5_12 2011-09-30 -- PRN24 clock predictions excluded from IGU combos due to UNUSUFN announcement, starting 1655_5_18 2011-10-05 -- IGU clock predictions restored for PRN08 after UNUSUFN outage, starting 1656_3_18 2011-11-04 -- SOPAC Data Center unreachable again from ~11:00 EDT (IGU 1660_5 _12) for >2d 12 hr; ftp connection restored before ~23:00 on 2011-11-06; SIO analysis products disrupted during the same time period 2011-11-08 -- PRN06 clock predictions excluded from IGU combos due to scheduled MX outage, starting 1661_2_18; also clock predictions restored for new SVN63/PRN01 (IIF-2) after being set healthy 2011-10-18 2011-11-09 -- PRN04 clock predictions excluded from IGU combos due to scheduled MX outage, starting 1661_3_12 2011-11-14 -- IGU clock predictions restored for PRNs 06 & 04 after MX outages, starting 1662_1_12 2011-12-01 -- minimum of 3 usable ACs (EMU, ESU, GOU) available for IGU GPS combo 1664_5_00 due to lack of submissions from CODE & GFZ; only 2 ACs (ESU, GOU) available for IGV GLO combo; GFU returned for next combo 1664_5_06 2011-12-09 -- IGU combo 1665_5_12 (but not IGV GLO part) delayed by 1 hr 12 min due to the starting ftp process getting hung; the ftp was manually killed & the process then completed normally (but late) 2011-12-15 -- unexpected orbit deviation by PRN25 occurred on 2011-12-10 that caused ~1 m level errors in IGU 1665_6_00 predictions; this was about midway thru eclipse season for this SV & could have been related to some attitude control issue 2011-12-19 -- PRN25 clock predictions excluded from IGU combos due to scheduled MX outage, starting 1667_1_12 2011-12-23 -- IGU clock predictions restored for PRN25 following scheduled MX outage, starting 1667_5_18 2011-12-30 -- ACC website pages updated to change originator to "IGS ACC" effective 2012-01-01 2012-01-01 -- Jake Griffiths replaces Jim Ray as ACC 2012-01-09 -- Kyle Snow (Topcon) reported a formatting error for the P flags in the IGV SP3 files; Jake corrected the bug that caused the error. 2012-01-15 -- perl module date_conv.pm in the tstacc account expired. The operational version was updated in November of 2011, but the test account was never updated. This caused the IGV products to fail. All missing IGV products were resubmitted. 2012-01-16 -- 16710_06 IGV re-resubmitted by mistake. 2012-01-19 -- added WHU to ftp getting processes for the ultra products and updated acchelp to begin looking for their products when the sub option is used. 2012-01-22 -- Begin excluding clock predictions for PRN03 due to MX event. 2012-01-27 -- Restored clock predictions for PRN03 following MX event. 2012-02-02 -- WHU began submitting product files to BKG, and the ACC processes retrieved them OK. 2012-02-03 -- when WHU began submitting files, the processes for the ultra caused that combination to try to use them. But, because the configuration files for the ultra did not include WHU, the combination didn't know what to do with the new submissions. Lesson: do not retrieve files unless the combination is prepared to use them. Resubmitted 1673_5_00 and 1673_5_06. 2012-02-09 -- clock on PRN01 suffers from apparently real glitch. 2012-02-09 -- PRN26 unusable till further notice; begin excluding clock predictions. 2012-02-13 -- accuracy code for PRN27 in whu16751_06.sp3 overflowed, resulting in "***" in the SP3 file, which violates the standards and causes the filtering of SVs with zero accuracy codes to fail. A symptom of this failure is that the SIO orbit (filtered just before WHU) was used also for WHU, causing the combination statistics for both SIO and WHU to be identical. The WHU group was contacted, and asked to use a 0 accuracy code in these kinds of situations. 2012-02-21 -- clock on PRN01 had several more glitches over the weekend; is now being excluded from the ultra clock predictions. 2012-02-27 -- NGS begins submitting ultra-rapid solution (NGU) for comparison only. The first contribution was for 16771_12, but the epoch records in their SP3 files were formatted incorrectly. The ACC rejects contributions if formats are incorrect. So, the NGU orbits were rejected and the ERPs were included for comparison. 2012-02-27 -- PRN25 unusable till further notice; begin excluding clock predictions. IGV used clock predictions till 28 Feb 2012 by mistake. 2012-02-28 -- PRN26 usable; stop excluding clock predictions. 2012-03-06 -- Restore PRN25 to clock predictions (in IGV, too) following UNUSUFN NANU. 2012-03-08 -- NGS made a change that should better align their predicted orbits to the IGR. This change became effective for the 1678_4_18 combi. 2012-03-09 -- Completed the additions to the ultra-rapid webpage for WHU and NGU. 2012-03-16 -- restored PRN01 to ultra clock predictions for 1679_5_12. 2012-06-18 -- modified IGV combinations so that the AC orbits are aligned using 24h of observed orbit instead of all 48h. 2012-06-24 -- IGV 1693_6_12 completely failed giving an error of unsupported SYS_TYPE. None of us had seen this before and the problem resolved itself by 1693_6_18. The error only happened in the IGV test environment, so there must be something special about that, but we cannot find it. 2012-06-28 -- verified that the Bernese control files had been updated for leap sec. 2012-06-28 -- informed SIO that their ultra-rapid orbits had very large rotational issues recently. The problem turned out to be related to the impending leap second. 2012-06-29 -- the NGU contribution to IGU 1694_5_06 had huge rotations. Notified NGS. It is probably an issue with the leap second. 2012-06-30 -- COU erps rejected during download because of an "unexpected jump in ut." It is unusual that CODE gets this wrong. After re-checking all areas for leap second tables missed before, found gpsutc file in .../common/orb_cnt/, which needed to have the leap second added. This did not seem to solve the problem because CODE still had bad orbits compared to the others. 2012-07-01 -- bug in ftp retrieval scripts caused complete failure in the distribution of AC products to the combination working areas when CDDIS was unavailable at midnight Saturday to Sunday (for the switch to new GPS Wk). This was the cause of the missing IGUs for 1695_0_00 and 1695_0_06 overnight. 2012-09-24 -- add WHU to IGV for comparison. 2012-09-26 -- add WHU to IGV with full weight. 2012-09-26 -- changed remote_daily_autocmb so that the logic assumes any AC may have mixed GLO+GPS SP3 files. Before, the list was hard-wired for ultras and rapids. 2012-10-07 -- switched ultras to IGb08, although the frame label in the combined SP3 file still says IGS08. This is consistent with most of the ACs reporting IGS08 frame still. 2012-10-08 -- switched rapids to IGb08; manually changed combined SP3 frame label. Still do not understand why IGS08 is being written to SP3 file. The Rapid PPP also was switched to IGb08 despite no new meta-data files for Bernese. This was achieved by processing only sites that are still useable for IGS08. 2012-10-09 -- found the "bug" causing IGS08 to still be written to the frame label in the combined SP3 files. The problem was that the new frame was written only to iers_frame.tab when it also needs to be written to itrf_frame_name.tab. 2013-01-02 -- turned off the testing in IGV; starting 1721_3_12, the GPS part of the IGV is identical to the IGU. The testing was to asses the impact of aligning AC orbits using the 24h observed part instead of all 48h. The results were inconclusive, probably because the IGU-IGR differences are dominated by EOP errors. 2013-02-28 -- three times in the past 4 weeks, outages at offsite location have delayed the ultra-rapid products. The outage today was caused by a failed power source knocking out internet connectivity. 2013-03-25 -- Begin excluding CODE from GPS ultra-rapids until rotation and ERP issues are resolved. At the same time, began including NGS PM & LOD and WHU PM. WHU LOD still looks biased. 2013-03-28 -- Three times in the past 4 weeks, outages at NRL have delayed the ultra-rapid products. The outage today was caused by a failed power source knocking out internet connectivity. 2013-03-28 -- P. Steigenberger noticed that the GLONASS ultra plots had not been updated in several weeks. Fixed. 2013-04-06 -- Planned power outage interrupts IGU/IGV and IGR product delivery. It was not known that the outage would cause Ken's network services to go offline. The ACC servers remained on because of the backup power supply, but they could not access the DCs. This has become a rather frequent issue as NRL upgrades main-line power in some of the buildings. 2013-04-08 -- Began excluding NGS LOD because it goes haywire from time to time. 2013-04-10 -- Switched to new ACC servers. This switch includes implementation of Bernese v5.2 for the brd2sp3, long-arc, PPN and PPP processes. The switch begins with IGU/IGV 1735_3_12, IGR 1735_2 and IGS 1733. 2013-04-24 -- Sharyl Byram (USNO) notified ACC about potential issue in downloading AC products when and AC resubmits. 2013-04-28 -- Another NRL power outage interrupts IGU/IGV and IGR products. Services were restored at about 21:30 EDT. 2013-05-16 -- Put in a fix to correct logic bug in ftp retrieval process spotted by Sharyl Byram (USNO) on 24 April. 2013-06-13 -- ESA excluded from Rapids and Ultras due to large RY (& PM-x) and RZ starting with 1744_4_12 (IGU) and 1744_3 (IGR). 2013-06-17 -- ESA restored to Ultras since other IGU errors are larger. 2013-07-10 -- USNO begins submitting USU 00h Ultra as USN Rapid. 2013-07-13 -- USNO stops submitting USU 00h Ultra as USN Rapid, even though they perform better. 2013-07-15 -- Restored CODE orbits starting with 1749_1_12. The ERPs were restored for one combination by mistake. 2013-07-23 -- recent chkcombo errors in ultra-rapids caused by incomplete GNSS SP3 files from WHU. Put in fix to address GNSS ACs who end up ^? submitting products with only one system. 2013-08-21 -- changes made 2013-07-23 killed the nanu checking in the ultras. This has been corrected by reverting to the old version of remote_daily_autocmb. 2013-08-24 -- ultra-rapid combination for 1754_6_06 got stuck in an infinite loop apparently trying to exclude COU and WHU for large transformations. This delayed the products for about 2h 13m. I interrupted the looping combination by killing it, and re-ran the combination with COU and WHU excluded a priori. The two ACs will remain excluded until their rotation issues are resolved. 2013-09-09 -- NRCan begins submitting GNSS ultra-rapid products starting with igu/igv combination for 1757_1_12. Their new GLONASS ultras receive full weight. 2013-11-12 -- CODE begins submitting 6h & 18h ERP Ultra-rapid files, starting with the combination for igu17662_18. The software change occured for igu17662_12. 2013-11-17 -- COU orbits included with weight. 2013-11-25 -- COU LOD added into combination with weight, starting with the combination for igu17681_12. 2013-12-09 -- clock predictions for PRN 24 added back into combination, should have been done when new IIF became usable. Currently, clock predictions for PRNs 01 & 30 are excluded from ultra combination.