Dev License: This installation of WHMCS is running under a Development License and is not authorized to be used for production use. Please report any cases of abuse to abuse@whmcs.com
Ares Outage (Resolved)
  • Priority - Critical
  • Affecting Server - ares
  • The Ares/Brists11 TeamSpeak node required a security patch to Teamspeak which unfortunately did not correctly take.

    We are working to resolve the issue and if we cannot we will roll back to the previous version.


    ---------------------------------------------------

    The issue has now been resolved and the node in now both running as well as updated.

  • Date - 06/01/2017 01:51 - 06/01/2017 02:46
  • Last Updated - 06/01/2017 03:46
Ares DDOS (Resolved)
  • Priority - Critical
  • Affecting Server - ares
  • The Ares (also known as brists11) is currently experiencing a DDOS attack beyond our ability to scrub continuously.

    For this reason the Ares node IP address has been required to be null routed by our upstream provider until it falls back within normal ranges of traffic.

    We apologise for any inconvienience caused to our clients and will restore the node asearly as possible.



    *NOTE: This node is now fully operational again, should you still experience any issues please open a support ticket.

  • Date - 06/06/2016 20:47 - 07/06/2016 15:00
  • Last Updated - 07/06/2016 16:09
Melbourne Data (Resolved)
  • Priority - Critical
  • Affecting Other - Melbourne Data Centre Traffic
  • We are receiving reports from users that large numbers of people are unable to connect to any node at the Melbourne Datacentre.

    Currently all nodes are online and operational and we are seeing some users connected so we are escalating the issue to our Transit provider to track down what/where the issue lies.

    If you are experiencing problems please let us know via ticket your:

    State:
    ISP:
    Traceroute results (from a windows command prompt "tracert melbts12.techsmith.com.au" replacing melbts12 with your node):

    We will provide further information as our transit provider updates us.

  • Date - 26/04/2016 20:39 - 26/04/2016 21:39
  • Last Updated - 06/06/2016 20:46
Melbourne Equinix Rack Moves (Resolved)
  • Priority - Critical
  • Affecting Other - All Melbourne Services
  • The above window has been allocated for our core network equipment to be relocated to our Equinix ME1 data centre.  During this time, there will be a full outage to our Melbourne network.

    This relocation will take approximately 1 to 2 hours to complete by our onsite network engineers.


    We do apologise for any inconvenience that this may cause.  If there are any questions regarding this notification, please contact our technical support team.

  • Date - 24/04/2016 23:00 - 25/04/2016 02:00
  • Last Updated - 26/04/2016 20:39
Sydts12 Node move (Resolved)
  • Priority - Critical
  • Affecting Server - sydts15
  • All clients on the Systs12 node have previously been made aware of this transfer although for those that would like to know at what stage we are up to we will keep this page updated.

    Database fully backed up
    Snapshots of TeamSpeak servers taken (as the backup, backup plan)
    Copied individual server data icons, files etc.
    Restored database and files to correct locations on new server.
    Updated database to newest TeamSpeak version
    Updating firewall rules
    Testing servers.
    Changing DNS record to point sydts12 to new server
    Old sydts12 node shutdown *currently in progress*

  • Date - 01/10/2015 09:00 - 01/10/2015 12:00
  • Last Updated - 24/04/2016 21:35
Mousai DDOS (Resolved)
  • Priority - Critical
  • Affecting Server - mousai
  • Our Mousai node is currently being hit with a complicated Denial of Service attack larger than we are able to mitigate, for this reason we have had to drop all traffic to this server in order to protect our other nodes and clients servers.

    We will work with our network provider to monitor the attack and negotiate with them a time when they will allow traffic to resume.

    As we get further information on estimated timeframes we will post it here.

  • Date - 25/03/2016 23:00 - 26/03/2016 08:00
  • Last Updated - 24/04/2016 21:34
All TeamSpeak Servers (Resolved)
  • Priority - Critical
  • Affecting System - TeamSpeak
  • The TeamSpeak team has made us aware of a vulnerability that exists in their client prior to 3.0.18.1 and has advised that clients prior to this may be open to having code executed on their machines if certain criteria are met.

    At their suggestion and for the safety of all our clients and their users we will be setting the minimum required client to connect to our servers to 3.0.18.1.

    These clients are now available for PC and Android although unfortunatly due to the approval process required by iOS it may take TeamSpeak up to a week for the update to be approved on that platform.

    *** All servers now require a Minimum version of client version of 3.0.18.1 to connect, this report will be left in progress for 24 hours so that it is easier for clients to find. ***

  • Date - 12/10/2015 10:00
  • Last Updated - 15/11/2015 12:08
IP Block via Transit provider (Resolved)
  • Priority - Critical
  • Affecting Server - ares
  • We have been made aware that our ARES TeamSpeak server is currently unavailable due to a block put in place by an upstream IP transit provider.

    The information we have received initially is only:
    ---
    Vocus has identified a potential threat towards your service and has taken action to mitigate this attack.
    Summary details of the event:

    • Source IP addresses: Distributed
    • Destination IP addresses: *Ares IP*
    • Time mitigation started: 18:22 AEDT
    • Triggered action: Remote Triggered Black Hole
    • Duration of mitigation: 24 hours
    ---


    At this stage we are unsure as to how far over our "scrub and soak" limit this attack is although as the block is outside of our network it is likely it will be beyond our control to bring this online any sooner than vocus has implemented.

    If any further news becomes available it will be updated here.

  • Date - 26/10/2015 18:22 - 27/10/2015 18:22
  • Last Updated - 15/11/2015 12:08
Large DDOS Attack (Resolved)
  • Priority - Critical
  • Affecting Server - apollo
  • We have been made aware that users are experiencing packet loss of close to 100% to our Melbourne servers currently.

    The issue was isolated to a large DDOS attack on our apollo TeamSpeak node at a consistent 4.2Gbit per second. This was exceeding our available allowance for scrubbing and soaking trafic which has resulted in it being null routed for 12 hours pending further checks in order to ensure our other Melbourne users are able to continue using services normally.

    We will review the incoming traffic with our transit provider in 12 hours with the aim to bring the node back online as quickly as possible.

    We apologise to any users on the Appolo node for the inconvienience.

  • Date - 03/10/2015 18:30 - 04/10/2015 00:00
  • Last Updated - 10/10/2015 12:21
TeamSpeak Traffic issues (Resolved)
  • Priority - Critical
  • Affecting Server - apollo
  • We are currently aware of issues with the Apollo TeamSpeak node and are currently investigating the cause of the problem.

    It appears after our first inspection that one particular TeamSpeak client on the node was receiving an attack designed specifically to trip certain safeguard alarms.

    We are testing the server being restored with the "culprit" clients server remaining down and will monitor to ensure other servers on the node start working as they should.

  • Date - 01/07/2015 21:08 - 04/07/2015 20:22
  • Last Updated - 01/07/2015 22:12
International Traffic (Resolved)
  • Priority - Low
  • Affecting Other - All international traffic
  • Our upstream provider has created an open window due to the tropical storm over Hawaii that may impact international connectivity to our network.  As indicated by the impact notice that we have received, the affected locations of Hawaii is The Big Island (the Spencer Beach Cable Station) and a lesser impact to Oahu (the Kahe Point Cable Station).

    The impact of this storm may affect the power to these facilities however as both sites have emergency plants and additional fuel supplies, no loss of power has been anticipated at this stage.

    Should any impact happen to our international connectivity, further updates will be announced.

  • Date - 08/08/2014 12:50 - 14/08/2014 00:00
  • Last Updated - 15/06/2015 15:50
Node Issue (Resolved)
  • Priority - Critical
  • Affecting Server - brits12
  • We are currently experiencing a failure with one of our Brisbane nodes, we have a technician onsite looking in to this issue and will post further updates as soon as they are available although at this point our technicians priority is on restoring services.

  • Date - 24/05/2015 07:30
  • Last Updated - 15/06/2015 15:50
Hazard Notification - 15/06/2015 - 16/06/2015 (Resolved)
  • Priority - Critical
  • Affecting System - Sydney Colocation/Dedicated Servers
  • The above outage window has been scheduled for upgrades to be carried out on our core network for dedicated server and colocation customers in our Sydney data centre.

    During this time, colocation customers will experience a brief 10 to 15 minute outage while network changes are completed.

    Dedicated Server customers will experience a longer outage while we carry out hardware upgrades to our core network.

    We apologise for the inconvenience that this may cause and if there may be any questions, please do not hesitate to contact our technical support team.

  • Date - 15/06/2015 22:00 - 04/07/2015 20:22
  • Last Updated - 15/06/2015 15:49
Melbourne Relocation (Resolved)
  • Priority - Critical
  • Affecting Other - Melbourne Services

  • Service Affected: All Melbourne Services
     
    Impact: Critical Impact

    Description:
    The above outage window has been scheduled for the 18th to the 19th of April due to relocating our services from the NextDC M1 data centre to the new Equinix ME1 data centre. 

    During this outage window, all melbourne services will be unavailable.

    Our technicians will assist with verifying that all services are available after the relocation has been completed.

    Services within our Sydney, Brisbane and Perth data centres will not be affected during the relocation of our Melbourne PoP.

    We apologies for the inconvenience that this may cause and if there may be any questions, please do not hesitate to contact our technical support team.

  • Date - 18/04/2015 22:00 - 19/04/2015 04:00
  • Last Updated - 24/05/2015 11:36
Database Error (Resolved)
  • Priority - Critical
  • Affecting Server - sydts13
  • A script/database issue on the new sydts13 server has caused TeamSpeak on the server to fail.

    As a consequence of this we were required to create a snapshot of all servers and move them into a new database.

    This process has now been completed and our monitoring scripting is now once again showing sydts13 as stable.

  • Date - 12/08/2014 17:35 - 12/08/2014 18:04
  • Last Updated - 12/08/2014 18:04
Brisbane TeamSpeak Hosts (Resolved)
  • Priority - Critical
  • Affecting Server - ares
  • Due to an critical change that is required to be made on our Brisbane TeamSpeak servers, we will be rebooting these servers to allow for these new changes to take affect.  During this time, all TeamSpeak servers will be shutdown and unavailable.  Our technical support team will ensure that all servers are powered up as quickly as able.

    Services in Sydney and Melbourne will not be affected by this outage.

    We apologise for the inconvenience that this may cause and if there are any questions, please contact our technical support team.

  • Date - 29/03/2014 15:00 - 29/03/2014 16:49
  • Last Updated - 08/08/2014 12:50
dDos attack (Resolved)
  • Priority - Critical
  • Affecting Server - sydts15
  • UPDATE: All services are now restored to the sydts12 server as of around 2:15am.
    ----------------------------------------------------------------------

    From around 3am we started receiving a growing amount of dDos attack traffic on this specific server. which at around 4:45am our datacentre could no longer mitigate and null routed traffic to that IP address.

    That IP address will be down for 24 hours although we are looking at spinning up three new servers today to further spread our traffic and lower the impact an attack has in the future, if you would like your account transferred onto a different Sydney server let us know and we will have it transferred over from yesterdays backup.

    We apologise for any inconvenience this causes.

  • Date - 16/07/2014 06:15 - 17/07/2014 02:15
  • Last Updated - 17/07/2014 06:13
Large DDoS attack currently effecting some se (Resolved)
  • Priority - Critical
  • Affecting Server - sydts15
  • **Update: Our traffic flow has been restored to bareable levels and services have once again been fully restored to Sydts12**

    We are currently under an extremely aggressive DDoS attack which has unfortunately finally proven that our Rackspace datacentre can take a hit from time to time as well.

    We have our NOC as well as the Rackspace NOC constantly monitoring the UDP flood in order to restore services as quickly as possible however we are unfortunately unable to offer an ETA on when this may occur.

  • Date - 17/06/2014 20:07 - 18/06/2014 17:49
  • Last Updated - 01/07/2014 17:44
Some services unavailable (Resolved)
  • Priority - High
  • Affecting Server - ares
  • He have been made aware that some services on the Ares server are experiencing loss of connection, we have investigated this remotely via our NOC and have dispatched a technician to the data centre in order to investigate the matter further.

    A processor on this server was causing the issues and services have now been restored.

  • Date - 02/05/2014 21:15
  • Last Updated - 02/05/2014 22:03
Brisbane Network (Resolved)
  • Priority - High
  • Affecting System - Brisbane Network
  • The above window has been scheduled for our network engineers to perform an upgrade to our core network in our Brisbane PoP.

    During this outage, customers on our Brisbane network will experience drop outs of up to 10 minutes at a time.  This upgrade will consist of new network infrastructure being installed into our Brisbane PoP as part of our extension of additional transit and peering providers being added to our network throughout the year.

    We apologies for the inconvenience that this may cause and if there are any questions, please contact our technical support team.

  • Date - 22/03/2014 22:00 - 23/03/2014 22:00
  • Last Updated - 29/03/2014 15:34
Sydney Website Traffic (Resolved)
  • Priority - High
  • Affecting Server - Hestia
  • Firstly this will NOT impact our Sydney TeamSpeak datacentre unless you are still on the "mousai" server, this will only potentially affect hosting and mousai server traffic.

    The above window has been scheduled by Vocus to perform a software upgrade to their border router.

    During this upgrade, an approximate 60 minute outage may occur due to our BGP session with Vocus being shut down.  

    As part of our redundant network that we have setup across all sites, any traffic that routes VIA Vocus will automatically reroute through to our other IP transit providers during this outage window.

    We apologies for the inconvenience that this may cause and if there are any questions, please contact our technical support team.

  • Date - 19/03/2014 11:30 - 19/03/2014 19:30
  • Last Updated - 29/03/2014 15:34
Melbourne Network (Resolved)
  • Priority - High
  • Affecting System - Melbourne Networks
  • The above window has been scheduled by Vocus to perform a software upgrade to their border router.

    During this upgrade, an approximate 60 minute outage may occur due to our BGP session with Vocus being shut down.  

    As part of our redundant network that we have setup across all sites, any traffic that routes VIA Vocus will automatically reroute through to our other IP transit providers during this outage window.

    We apologies for the inconvenience that this may cause and if there are any questions, please contact our technical support team.

  • Date - 19/03/2014 14:00 - 19/03/2014 18:00
  • Last Updated - 29/03/2014 15:34
Hazard Notification (Resolved)
  • Priority - Critical
  • Affecting Other - Upgrade to Melbourne Network
  • We will be performing an upgrade to the core network in our Melbourne PoP during the above window.  During this time, services (shared hosting, virtual servers, dedicated servers TeamSpeak & colocation) in our Melbourne PoP will be unavailable.

    Services to our Sydney and Brisbane PoP will not be affected by this outage.

    We apologies for the inconvenience that this may cause and if there are any questions, please contact our technical support team.

  • Date - 13/02/2014 22:00 - 14/02/2014 02:00
  • Last Updated - 25/02/2014 22:23
Hardware Upgrade (Resolved)
  • Priority - High
  • Affecting Server - ares
  • You may have recently heard that we are upgrading our infrastructure and as part of our upgrades, we are now ready to start migrating Brisbane TeamSpeak servers across to our new infrastructure.
     
    The new DELL Enterprise servers come with the latest technology offering Intel E5 processors, multiple gigabit network ports, DDR3 ECC memory &  RAID 10 hard disk redundancy.

    The migration for Brisbane TeamSpeak server(s) will kick off on Friday, 31st January, 2014 after 9:00PM (Brisbane, QLD Time).
     
    During the migration process, servers will be shutdown and suspended while we move the images across to our new DELL Enterprise servers.  Once the images have been migrated over, our technician will re-enable the server and bring them back online.

    During this migration process, your IP address/connection info will remain unchanged.

    Please feel free to contact our technical support team by raising a eTicket with them if you might have any questions about this migration.

    *******************************************************************************************
    Work has been completed and all TeamSpeak servers have been successfully migrated over to the new Hardware.
    *******************************************************************************************

  • Date - 31/01/2014 21:45 - 31/01/2014 22:45
  • Last Updated - 01/02/2014 00:06
DDoS Attack (Resolved)
  • Priority - Critical
  • Affecting Server - apollo
  • Our Apollo server is currently under heavy DDOS attack, Unfortunately in order to protect the rest of our network as well as to comply with our upstream provider policies we have had all traffic to this IP address temporaraly null routed for 24 hours.

    We are sorry for any inconvienience this may cause and thankyou for your patience.

    To answer a couple of questions we have been receiving:

    Yes, our Sydney TeamSpeak servers are all being spun up on a premium upstream provider, currently at no additional cost, which has been at 100% uptime now for three months, although theoretically enough bandwidth could be generated in a DDOS attack to effect this network also.

    Yes, we can do quotes on setting up and maintaining your own private VPS, IP address, and TeamSpeak server to seperate your traffic from other TeamSpeak servers on a node, several larger clans have allready been doing this with us for some time, this would mitigate the effect of a DDOS so long as your server was not the specific target. Simply open a sales/billing ticket and let them know you are interested and they will give you an idea on pricing unique to your needs within 24 hours.

    ********************************************************************
    Full traffic has been restored, if you are still experiencing issues please open a ticket.
    ********************************************************************

  • Date - 28/01/2014 23:24 - 29/01/2014 21:58
  • Last Updated - 29/01/2014 22:59
Apollo Packet Loss (Resolved)
  • Priority - High
  • Affecting Server - apollo
  • Our NOC is currently investigating what is causing some of our users to be experiencing roughly 20% packetloss at the moment.

    The issue has been traced to an upstream providers equipment which has been causing the problem for some users, we have now been informed that the equipment has been  replaced and effected users have been reporting they are no longer noticing the packet loss issue.

    If you are still effected please open a support ticket and we will investigate further with you.

  • Date - 14/01/2014 10:46 - 14/01/2014 11:45
  • Last Updated - 14/01/2014 14:17
TeamSpeak 3.0.10.2 update (Resolved)
  • Priority - Low
  • Affecting System - All TeamSpeak 3 Servers
  • Teamspeak today announced a "hotfix" patch (3.0.10.2), we have all servers allready prepared for the update and will undertake a rolling restart on server Friday to bring all servers up to the latest version.

    Estimated downtime per server is less than two minutes.


    -----------------------------------------------------------

    All servers were updated successfully with no server requiring extended maintenance, average downtime for each server was less than 20 seconds.

    -----------------------------------------------------------

  • Date - 13/12/2013 11:00 - 13/12/2013 11:10
  • Last Updated - 13/12/2013 11:35
TeamSpeak Server Version Update (Resolved)
  • Priority - Low
  • Affecting Server - ares
  • We will me applying the 3.0.10.1 update to all current Teamspeak servers on this server on the 5th of December, prior to this we will be running a full scan on the drives so exact time of upgrade at this point is unknown although downtime based on other server upgrades should be between one to five minutes.

    Ares scan took us dangerously close to peak usage time so we decided to push the upgrade for this server over to the 6th, the update has now been complete and all of our TeamSpeak servers are now running version 3.0.10.1

  • Date - 05/12/2013 12:47 - 06/12/2013 09:24
  • Last Updated - 13/12/2013 00:16
Large DoS effecting our IP Transit provider (Resolved)
  • Priority - Critical
  • Affecting Server - ares
  • Our NOC has been informed that our Upstream provider is currently attempting to mitigate a large DoS attack, whilst they are working to resolve this as quickly as possible they have not yet given us an ETA.


    Update: We have been informed that traffic to targetted parties has now all been successfully null routed and our monitoring shows conectivity across Brisbane servers has been restored.

  • Date - 07/12/2013 10:40 - 07/12/2013 19:52
  • Last Updated - 07/12/2013 20:56
TeamSpeak Server Version Update (Resolved)
  • Priority - Low
  • Affecting Server - mousai
  • Update: All TeamSpeak servers on this server have now been upgraded to version 3.0.10.1, downtime was around two minutes.

  • Date - 05/12/2013 12:44 - 05/12/2013 12:44
  • Last Updated - 05/12/2013 13:44
TeamSpeak Server Version Update (Resolved)
  • Priority - Low
  • Affecting Server - apollo
  • Update: All TeamSpeak servers on this server have now been upgraded to version 3.0.10.1, some changes on this server required a reboot, downtime was around eight minutes.

  • Date - 05/12/2013 12:43 - 05/12/2013 12:43
  • Last Updated - 05/12/2013 13:44
TeamSpeak Server Version Update (Resolved)
  • Priority - Low
  • Affecting Server - sydts15
  • Update: All TeamSpeak servers on this server have now been upgraded to version 3.0.10.1, downtime was around two minutes.

  • Date - 05/12/2013 12:40 - 05/12/2013 12:40
  • Last Updated - 05/12/2013 13:42
Unable to connect issues. (Resolved)
  • Priority - Critical
  • Affecting Server - ares
  • He were made aware of an issue with people being unable to connect to the Ares server earlier today and it has now progressed to being generally unresponsive to all traffic but ping.

    We have escalated from remote support to hands on and hope to have answers and services restored as quickly as possible.

    A technician diagnosed a potentially corrupted drive as the cause of the issue and took steps to rectify the problem.

  • Date - 26/11/2013 17:33 - 26/11/2013 19:33
  • Last Updated - 05/12/2013 13:40
Billing and automation software upgrade (Resolved)
  • Priority - Low
  • Affecting System - Billing and Automation System
  • Our billing and automation software provider has recently released both a feature and security patch.

    Judging by our test server update we expect that applying the upgrade will not cause more than a five minute disruption to the client area and ordering systems.

    If we come across any complications we will update this page.

  • Date - 28/07/2013 09:50 - 28/07/2013 09:55
  • Last Updated - 22/11/2013 17:44
IP Transit outage (Resolved)
  • Priority - Critical
  • Affecting Other - Sydney Services
  • Last night our IP Transit provider became unreachable for both traffic as well as support which has caused us to bring forward plans to move our teamspeak services in Sydney to a new Datacentre/IP Transit provider.

    We were going to do this in stages and transfer clients across in blocks however given the recent instability we have decided that we will send a bulk email out to all people with effected services this afternoon and give everyone the opportunity to transfer now.

    If you have not received an email by Tuesday and you have a Sydney based teamspeak service please open a support request and we will send you an update.

  • Date - 27/10/2013 22:25 - 28/10/2013 02:30
  • Last Updated - 28/10/2013 07:30
Website system upgrade (Resolved)
  • Priority - Low
  • A potential security vulnerability was attempted to be exploited earlier today and although no unauthorised access has been gained we have rapidly deployed a security patch to prevent further atempts at a similar exploit.

    During the upload of the patch several files needed to be modified which resulted in periods of downtime to our website only, no client products or services were affected throughout this upgrade.

  • Date - 23/10/2013 12:23 - 23/10/2013 13:24
  • Last Updated - 23/10/2013 14:27
Server Outage (Resolved)
  • Priority - Critical
  • Affecting Server - mousai
  • Clients are currently experiencing an outage on the Mousai server.

    Early trace routes suggest All traffic is routing to Sydney Equinix datacentre and not progressing any further.

    We have network engineers looking into the cause of this outage.

    ---------------------------------------------------------------------------------------------

    The outage has been traced to a Denial of service Attack on the mousai IP address which has caused the datacentre to null route this IP address.
    We have been informed that they will not release the null route until around 1am on the 9th of September so long as the ammount of traffic no longer negatively impacts their network.

    Unfortunately this has been beyond our control however if you wish you may open a support ticket requesting we create a temporary server on either our Brisbane or Melbourne locations at no cost to yourself.

    We are also looking into options for the future to mitigate a future occurance of this happening such as running a redundant backup server off a seperate IP address as well the potential to leave our current upstream provider and move to one who will take a more suitable approach towards mitigating any future DOS attacks.

    ---------------------------------------------------------------------------------------------

    Services have now been restored.

  • Date - 08/09/2013 04:10 - 09/09/2013 01:00
  • Last Updated - 09/09/2013 01:08
Apollo outage (Resolved)
  • Priority - Critical
  • Affecting Server - apollo
  • We are currently investigating why the Apollo server is not currently responding to internet traffic.

    The server is still reachable via an internal path and is show all hardware as fully operational so we are now working with the datacentre to find the cause of the issue.

    --------------------------------------------------------------------------------

    All traffic flow throughout the datacentre has now been restored.

  • Date - 11/08/2013 23:00 - 12/08/2013 08:00
  • Last Updated - 12/08/2013 11:30
DOS Attack (Resolved)
  • Priority - High
  • Affecting Server - ares
  • We have been made aware that a segment of users is currently unable to connect to the Brisbane datacentre.

    We are looking into the cause of this issue and will update as further information arises.

    Update: It appears that the IP address assigned to the ARES server is now receiving no incoming traffic, we are looking into whether this is a malicious attack or due to other causes however in the meantime we have changed the effected servers IP address and updated DNS records.

    Your servers are now reachable again and should require no reconfiguration if you are using ares.techsmith.com.au to connect.

    The cause of this was a Denial of Service Attack, we have forwarded available information on to authorities.

  • Date - 11/07/2013 19:27 - 12/07/2013 19:27
  • Last Updated - 15/07/2013 20:57
Upstream Provider DOS (Resolved)
  • Priority - High
  • Affecting Server - apollo
  • At around 2200h we were alerted to random connectivity issues which quickly escalated into all traffic to the Melbourne datacentre failing.

    Our upstream provider has informed us:
    "We have had a couple of DOS attacks and we have now taken steps to null route them. 
    The issue is now fixed. We are still monitoring services for any further issues."

  • Date - 01/07/2013 22:06 - 01/07/2013 22:30
  • Last Updated - 01/07/2013 23:12
Upstream provider outage (Resolved)
  • Priority - Critical
  • Affecting Server - mousai
  • At around 2200h a hardware failure in one of our upstream providers routers caused us to go dark in Sydney.

    The issue has now been resolved and we are monitoring with them to ensure all services have been restored.

  • Date - 25/06/2013 22:00 - 26/06/2013 06:00
  • Last Updated - 26/06/2013 07:57
Upstream provider emergency upgrade (Resolved)
  • Priority - Critical
  • Affecting Server - Hestia
  • At around 2200h a hardware failure in one of our upstream providers routers caused us to go dark in Sydney.

    The issue has now been resolved and we are monitoring with them to ensure all services have been restored.

  • Date - 25/06/2013 22:00 - 26/06/2013 06:00
  • Last Updated - 26/06/2013 07:56
Server connectivity issues (Resolved)
  • Priority - High
  • Affecting Server - ares
  • We have been made aware that several users are experiencing intermittent outages on the Ares TeamSpeak server.

    Upon investigation the server itself is still running without issue and for most users they are not experiencing any issues.

    For those that are we are working with our upstream provider to try and determine where (and on whose network) the problem resides to we can request that they fix the issue.

    In the meantime if you are experiencing outages please try and capture an unsuccessful tracert and send it to support@techsmith.com.au so we can more quickly identify where the problem lies.

    For most of our users to complete the tracert you will need to:

    Press "Windows Key"+R
    type cmd.exe
    press enter
    type tracert ares.techsmith.com.au
    press enter
    Copy and paste the output into an email.

    We will update this page as more information becomes available.

    ------UPDATE-------
    The issue has been traced to an external network segment who once made aware informed us they were working to resolve the issue as quickly as possible.
    Since the update from them service seems to have stabilised for several hours now, however we have now put more permanently put into place overseas monitoring of our servers as well as our local and cross datacentre monitoring.
    This will give us quicker response to external network issues unrelated to Techsmith which may be impacting on a portion of our clients rather than our services directly.
    ----------------------

    We still ask that if you are still receiving outages to contact us to ensure we keep pushing on the source of the issue until the problem is fully rectified.

  • Date - 13/06/2013 12:37 - 13/06/2013 18:33
  • Last Updated - 13/06/2013 18:35
Mousai TS3 server upgrade (Resolved)
  • Priority - High
  • Affecting Server - mousai
  • We have been made aware that the mousai server originally seemed to display characteristics of a DDOS attack however on further investigation have found the cause to more likely be hardware related.
    We are currently monitoring the server as well as preparing hardware to fail over to in order to run further diagnostics no the server.
    Any downtime should be minimal so long as the hardware issue stays at it's current level and we apologies for any random disconnects you may experience.

    The new Sydney Teamspeak server has been built and all servers have had snapshots taken and copied accross to the new server, we are monitoring for a low to zero activity period in order to switch ip addresses of old and new servers so as to ensure no downtime to clients.

    ETA for the cut over of servers is 16:30, estimated downtime will be from 5-15 minutes.


    All change over has been complete, no changes are required on our clients end from this moment on when you connect to mousai you will be connecting to the upgraded server.

  • Date - 18/05/2013 23:57 - 21/05/2013 16:28
  • Last Updated - 21/05/2013 16:30
Invoice emailing (Resolved)
  • Priority - Low
  • Affecting Other - Billing Services
  • He have been made aware that for some users, invoices have not been generated after the first month of having a service with us.

    The billing itself for 99% of clients has been processing correctly so for all but a very small number of clients the only thing you will notice in the next couple of hours will be a number of invoice emails being generated and sent out as our system "catches back up".

    The glitch appears to have been caused by a slight scripting change between major versions of our billing software and we are looking into the exact reasons why only some clients have been effected.

    If you find any irregularities occuring over the next few hours that concern you please don't hesitate to open a support ticket and we will answer your questions.

  • Date - 16/05/2013 16:54
  • Last Updated - 19/05/2013 00:02
Billing/Support corruption (Resolved)
  • Priority - Low
  • Affecting System - techsmith.com.au
  • We recently received an email from one of our providers (whmcs) stating they had been made aware of a security issue with their software (which is normal) and provided us with a patch to fix the issue.

    Ironically three hours before applying the patch our website mysteriously became corrupted, our logs show that no client data of any kind was accessed simply our website was no longer viewable.

    We took this opportunity to bring forward an upgrade to the software which runs our website so that now as we develop further applications to add to the user area we will no longer have to schedule downtime in order to bring them live.

    Once again we would like to stress that this issue did not cause any outage to any clients other than to the techsmith.com.au site being able to be displayed.

  • Date - 14/03/2013 17:33 - 14/03/2013 19:34
  • Last Updated - 14/03/2013 20:39
Melbourne POP (Resolved)
  • Priority - High
  • Affecting System - MELBOURNE
  • The outage that was to occur on the 1st of March has been rescheduled to the 6th of March between 10pm and 02am as we have received confirmation that the technical issues within NextDC M1 datacentre have been resolved.

     

     

    -----------------Original scheduled outage Information------------------------

    Our relocation of our Melbourne POP will take place on the 1st of March.

    During our move from the Pipe Networks data centre to the new NextDC M1 data centre we will also be adding additional infrastructure to our core network in Melbourne.

    During the outage, all technicians will be on site and we aim to minimise the downtime as much as possible.

    All customers in Sydney & Brisbane will not be effected by our Melbourne outage.

  • Date - 01/03/2013 22:00 - 07/03/2013 02:00
  • Last Updated - 07/03/2013 18:03
Payment Gateway (Resolved)
  • Priority - Low
  • Affecting Other - Paypal

  • A recent change in our plan naming saw a perfect time to do a couple of upgrades on our websites processing systems.

     

    Unfortunately as per all "upgrades" you run the risk of things not quite going the way they did in the lab, in this case our payment gateway refused to... well open the gate resulting in any new orders or manual payments seeing a blank page instead of the usual paypal site.

    This has NO impact on any client services however if you were trying to add credit or purchase new services you now know why.

    We will reboot apache on the Hestia server at an estimated 23:00 to rectify the issue.

  • Date - 02/01/2013 04:07 - 03/01/2013 00:11
  • Last Updated - 02/01/2013 21:14
Data Center Move (Resolved)
  • Priority - High
  • Affecting System - Techsmith Network

  • During this outage window, all services on the Techsmith network will be shutdown while we re-locate the equipment that host our virtual servers from the Global Switch data centre to the new Equinix Syd #3 data centre. During this move, our core network infrastructure will be shutdown and moved as a priority to bring our connection back up.

    Once all of the equipment has been moved to the Equinix data centre, our services will start to resume back to normal.

  • Date - 24/11/2012 13:00 - 23/02/2013 10:09
  • Last Updated - 22/11/2012 18:34

Server Status

Below is a real-time overview of our servers where you can check if there's any known issues.

Server Name HTTP FTP POP3 PHP Info Server Load Uptime
apollo PHP Info
ares PHP Info
brits12 PHP Info
bts02 PHP Info
bts03 PHP Info
Hestia PHP Info
melbts12 PHP Info
melbts13 PHP Info
melbts14 PHP Info
mousai PHP Info
perthts01 PHP Info
sydts13 PHP Info
sydts14 PHP Info

Powered by WHMCompleteSolution