The online UPS’s used in our BOL1 datacentre will be taken out of circuit tomorrow for a short maintenance period.
The associated Lead Acid batteries are being replaced to ensure extended contingency cover can be provided during any switchover phases between Utility Mains and Local Generation, eg: in the event of any Utility failure. This planned maintenance should not affect any hosted services, but will mean that some services will be subject to minimal ‘elevated risk’ for the duration of the maintenance.
Further notices will be posted during the maintenance window.
This morning at approx. 8:55am, our BOL23 site saw a momentary loss of mains power from the electricity grid.
As a result, our standby generators started up in preparation to take the power load from the UPS supplies. The grid power came back online after a few seconds so no transfer to generated power occurred.
The generators continued to run for several minutes in case of a further loss of mains power, but was un-needed.
They automatically shutdown and returned to standby status.
No loss of power to Netnorth or customer equipment occurred and all services remained online for the duration of the grid power failure.
We experienced another power outage of phase L3 in our BOL2 datacentre this morning at 3:09am.
Electricity Northwest are currently on-site tracing an underground cable fault.
It is hoped it will be within a section that can be isolated to restore power to the building.
The datacentre is currently running from generated power, and no outages are expected.
Staff are onsite monitoring the generators to ensure continued service.
Further to yesterday’s outage, we were alerted to another identical outage this evening at 11:26pm.
Once again, our automated failover systems migrated the site to locally generated power.
We are awaiting an update from ENWL, our regional energy supplier, with regards to a time to repair.
We experienced a power outage of phase L3 on the commercial mains input at our BOL2 site at approx. 5:45pm this afternoon.
The automated failover procedure occurred as expected and the site transferred over to locally generated power. The entire site became powered by generator for the duration of the phase outage.
We will continue to operate under generator power until all phases are within specification.
At 11:59 the commercial power was restored to all 3 phases.
This was detected by our switchgear, and after an observed period of stability, the generator was put on standby and normal service was reinstated.
We do not expect any further interruption, but will continue to monitor the service.
Circa 10:46 one of the three commercial phases to the Queensbrook estate failed somewhere upstream of the on-site distribution.
This supplies our BOL1 U8 datacentre and our administrative offices, no other Netnorth datacentres or infrastructure have been affected.
The partial supply failure was identified by our switchgear, and the standby generator was spun-up and brought online within 9 seconds. At all times our datacentre services remained live via our online UPS systems. Any equipment located in the affected datacentre which operates with dual feed power will indicate commercial power failure but will continue to operate on the protected supply.
Our utility provider has been informed and will attend to the problem off-site.
We will advise of any further developments.
At 8:37pm, switch 23 went offline due to a power failure.
Engineers attended site and diagnosed the cause of the power failure and re-established power at 10:28pm and will continue to monitor the power for the next 24 hours.
Any customers connected to switch 23 will have seen a network outage during this period.
At approximately 7:10am this morning, we experience a commerical power failure of the L2 phase of power entering out Bolton 1 facility.
This caused our Generator to take over the entire load of the building.
Cooling was maintained via ‘free cooling’ (external filtered air travelling through the building via powerful fans) – temperature and humidity were maintained at their normal levels for the duration.
Electricity North West were contacted and estimated a time to fix of 11:30am at the latest.
At approximately 10:15am, power was restored to all three phases and the Generator handed the load back over to commercial supply.
Electricity North West contacted us by telephone to confirm the supply was operating normally again.
Our UPS batteries are currently at 98% and recharging, and Generator is back on standby.
This power outage did not affect the connectivity of the Bolton 1 facility.