Just want to get over and done with the compulsory census and continue on with my online shopping.
The census website is overloaded. Server is down. So annoying!
Your thought/tips.
ABS Census Website Overloaded
eatwell365 on 09/08/2016 - 20:33
Last edited 10/08/2016 - 04:44
Last edited 10/08/2016 - 04:44
Related Stores
Comments
Now ABS claimed that it was 4x DoS attack from overseas. Not surprised if it's indeed some hacker wannabes launching LOIC on census day to make a name for themselves, but ABS should have done more.
what could be the next hypothetical excuse…. because of the Olympics Gold.
Sorry just not good enough to gain our trust in my opinion.
ABS (All BS) exactly!!!
I did hear that if you swallowed anything during the census that you could experience sudden explosive diarrhea…Blame it on Rio!
I believe the census was implemented by IBM using their cloud technology at a cost of nearly $10 million. Quite the big fail. Can't help but wonder if Amazon or Microsoft would have done a better job, both having technologies that are designed to scale.
Huge fail, but pretty typical in today's world of outsourcing to "cheap" providers. I work in this field and this is pennies compared to what I see "fail" on a daily basis due to outsourcing to big companies with most of the staff located in the 3rd world (like IBM).
And what a cop out today about the denial of service attacks. If they allowed that to happen in this day and age then whomever setup the security should be sacked on the spot. I find it very hard to believe in this day and age, sounds like shifting blame and excuses to me.
I got worried the moment I saw it was a jsp site. What year do they think it is 2005?
I'm certain it would have gone better on Azure, even without a lot of effort using their auto-scaling.
What makes you think IBM doesn't have auto scaling. Looks like ABS undermined in capacity planning!
Asked about the ability of the online census database to cope with such heavytraffic, an ABS spokesman said the site could handle "1,000,000 form submissions every hour. That's twice the capacity we expect to need"
Since everyone was told it is Census "night" and there is $180 fine per day, most people (millions) tried to submit that evening itself! For eg: 1m capacity, but 5 or 10m online!
They expected 500k responses an hour, engineered a system to handle up to 1m, then asked the 15m online respondents to complete the form on one night.
They Denial of Serviced themselves.They paid nearly $500,000 to Revolution IT to load test, so one will have to assume that they did not set realistic parameters for load testing …
ABS would of provided the requirements. Revolution It would of have to of had that to response to the tender.
They should really have DDoS protection.
In their defence, I understand that they were getting hammered by hackers at IP 127.0.0.1 ;)
I reckon it's the standard "OMG… its a DDoS, from.. ummm…China. Yeah it's APT-1." bollocks.
I'm surprised that they didn't use some standard DDoS prevention services like Akamai or Cloudflare.
Can they get away with that statement blaming hackers not them? Was it IBM fault or such attacks could not be tackled by IBM or any IT company?
I find it quite ironic that people are complaining about a possible breach of privacy…. on FACEBOOK. Zuckerberg has more personal information on you than you could ever imagine.
Only if you choose to give it to them. The census is compulsory.
If you think Facebook is only gathering information that you have authorised, you'd be mistaken.
Facebook can't fine me $180 a day until I send them my info.
Not all of us have Facebook.
Ah hence the irony is referring to Facebook users.
It's also voluntary, there are no criminal penalties for entering incorrect data and unless things have changed you can request your data to be deleted.
@fruit: No the act says they can compel you to answer accurately and threaten you with fines for not doing so.
@fruit: My apologies.
… but give it a few years and the gubmint will allow fb to force you to bend over as far as it wants you to.
I think their real issue is irrational distrust of government and big corporations. Privacy is just the excuse.
Why is distrusting government and big corporations irrational?
Following recent (wikileaks, Snowden, Four Corners, for example) revelations, blindly trusting governments and big corporations is incredibly foolhardy.But fair enough. I am from the government/big corporation. Kindly direct message me all your credit card numbers and I'll tell you if any of them are lucky.
You say that "Privacy is just the excuse" but actually it is as simple as "not trusting government with Privacy", so your comment doesn't really make sense… it's not an excuse for anything.
As far as being irrational, well our Governments (and Companies, Non-Profits, Individuals, etc.) are always having data leaks. It's just the reality of information security (not matter who the organisation is) that nothing can ever be 100% secure. Promises are not guarantees, it will leak anyway and then they will make excuses for it.
Our Government and our allies can't even keep a lid on the coverups they are doing without them leaking… why would you trust them with your data.
They say that the personal details will be kept safe and then deleted 4 years later. Where's the oversight to monitor that they are keeping to their word? They say that ASIO and other intelligence organisations aren't allowed to get a copy, but what's to stop them from just grabbing a copy anyway (or them being the hackers themselves) despite the law. It is now a known fact that intelligence agencies actually do that. We are in an age of terrorism and our Government uses that as a blank cheque to do what they like even if it's against the broader public interest.
irrational? Why? Does personal info no longer get duplicated, traded, shared and abused on the Interwebs now?
Okay can you think of one example where this has happened in Australia by a Government agency or large company, without having to Google it?
When it does happen it was never intentional.
@inherentchoice:
I work in telecommunications, so all the examples I can think of start there:
- Vodafone store staff shared passwords to online systems that made customer data available to non-employees
- Telechoice dumped paperwork in an unattended shipping container on a vacant lot that was then opened by vandals who left customer's details lying around on the ground
- AAPT had a customer database on a web server hacked exposing business customers details.As for government, the Police service has been in trouble for officers searching for celebrities, ex-romantic partners and other use of their systems without valid reasons. The QLD Health service had employees doing the same.
And the Snowden files showed systemic sharing of intelligence collected amongst the 5-eyes nations, including many instances of information collected on people who had no connection at all to the targets of the surveillance. Similarly the QLD doctor from Pakistan who was wrongfully imprisoned for a tenuous connection to a SIM card he gave to a relative that was subsequently linked to a terrorist incident thanks to information shared from British authorities.
That's half a dozen accidental and deliberate instances off the top of my head. Apologies if the detail is missing, as I couldn't use Google!
Your defence of privacy invasions is very ill considered.
@mskeggs: Thanks for the examples.
So by looking at the breaches that have occurred, I think it's irrational to distrust the government to the point of withholding your name in the census based on some isolated mistakes, hacking or inappropriate snooping.
The privacy principles provide specific exemptions for ASIO/law enforcement agencies to share intelligence, so that's not technically a breach.
I'm not sure which privacy invasion you think I'm defending? It may be an invasion of privacy for the Census to collect names - I wasn't intending to defend that. But regardless of whether or not the Census should collect names, I'm saying it seems irrational to me that some people/politicians are so opposed to providing their own name to the Census, especially considering all the other times when their name is disclosed.
What's gonna happen if the Chinese government obtains all the census data including names, addresses and current incomes? Genocide?!?
@inherentchoice: Yes, it has happens here. All the time. No organisation is safe. Average time before an organisation realises it has been compromised is >2 years. Most never admit it publicly. Least of all governments. Occasionally customers find out, mostly of the time that's how it becomes public.
If you this is news to you, try searching for breaches. This one is for 2015: http://www.itnews.com.au/gallery/the-biggest-australian-data…
Bare in mind that most go unreported. And most large networks are permanently hacked/cannot rid themselves of unauthorised activity.
And if you think we are not a target here in Oz, our Internet connects all our organisations to the rest of the world. Creating honeypots of data for all the miscreants to plunder is just so 1990s. Esp. not poorly protected, valuable ones.
The gov is approving and encouraging depts to spend public money without mitigating, or even understanding the risks involved. The state of ABS IT in 2013 is well documented and abysmal. Desperate management know the only future they have is in selling our information and in supporting the gov to retain power as there is no other way to get a promotion or even an above average performance review. Yes there are many good uses for census data, but the more the dataset gets personal, the more risk the population faces and the less people will provide accurate info, or contribute at all, despite the smiles, threats and fines.
Anyhow, what has intentional got to do with it? Incompetence and the realities of Internet security around big data sets is all that counts here. Not to mention trashing >5 years of work and the data that could have been available if the gov had simply done another Census just like it did in 2006.
What's gonna happen if the Chinese government obtains all the census data including names, addresses and current incomes? Genocide?!?
If you were originally from China, living and working in Australia (perhaps even a naturalised citizen here), but you still have folks back in China. Depending on the information provided, it could be put additional scrutiny on your family. Remember the ramifications of corruption allegations/crackdown in China are not the same as the Western world (the equivalent of a disapproving look).
Now replace China with North Korea, still seems pish posh to you?
Without elucidating further, but the census information provided is sufficient to perform social engineering attacks while masquerading as the victim, and will work surprising well for a number of organisations/service providers.
Edit: Shite grammar/missing words
@inherentchoice: Tese are not isolated mistakes. They are the norm in the always connected age.
There are far too many problems and it is getting exponentially worse. E.g. The ANAO reported in 2014; "The Australian Signals Directorate (ASD)3 has estimated that between January and December 2012, there were over 1790 security incidents against Australian Government agencies". This audit also noted that Security Controls were not in place at the ABS and it was also the least secure of all the gov agencies handing Personal information.
With the impacts of breaches being uncertainty at least, ID theft and you name the sky and the planet its around at worst, the reactions you are observing in some are nowhere near irrational- they are entirely predictable. And we should be nervous, there are no practical protections for privacy in Oz, none. Courts are nowhere near across the risks and focus on impacts that can be proven. We only have a toothless privacy commissioner (He even came out and supported/approved the ABS efforts regarding privacy within 24 hours of the debacle starting), yet this census breaches the APPs by not giving people access/ability to see and update their data, let alone the collection and storage of it when it is not necessary.
And yes, if the Chinese invade, they will use all the info they collect from the gubmint to work out who has money, who disagrees, who to watch, etc. They have very skilled people working on extraordinary datasets, just as we do. Now they shouldn't invade, but we are likely to face turmoil in our region and wars have changed the world many times before. Unexpected outcomes need to be expected. Imagine a world in which Trump decides to save money on NATO and ANZUS support.
And if the Chinese don't invade, it may be possible that the Japanese do, or the Americans. All long shots… but, if someone accesses the data any one of these (or our local) parties have been collecting, and figure from your families' activities that Auntie Noonie has been collecting money under her bed, they could come calling. Thieves and organised crime gangs, let alone gubmint depts, are modernising and they are very adept at following money trails.
Both are moot however as things are getting so bad now. We already need to be careful who is watching/assessing/analysing, and noone seems to know who is watching (or hacking) them, or when all the data they store might get mixed up, throw false positives and start flagging Citizens for interrogation under the draconian laws that now exist. Oz has no 2nd ammendment. We have no right to privacy.
We do need start caring before they come knocking like the Nazis did for the Jews in Holland, Freedom is not easily fought for and won- it takes generations to win, but is lost in an instant.
@mskeggs: Here's another one - the Department of Immigration and Border Protection which admitted to inadvertently leaked the personal details of close to 10,000 asylum seekers housed in Australia via its website. http://www.itnews.com.au/news/immigration-dept-confirms-asyl…
Daniel of service attack is not hacking. It's just an attempt to make the service unavailable for users. But that's the basic thing any online website will be prepared for. If they can't handle this how they are going to handle the complex hacking attempts.
Or its just everyone logging onto the lodge their census at the same time… and they calling it a DDOS…
These days, your data (personal information) is worth gold in terms of marketing, advertising, and sale to large institutes for "security" and other reasons.
So it's no surprise that the ABS was hacked as it's seen as a a golden treasure chest for hackers to access 1 site with all our personal info.
this is why so many of us have railed against the ABS in requesting our names on the forms
I wonder how they are going to identify the individuals who haven't completed the census.
To avoid exposing my secretive data , if hacked
What if I enter slightly wrong date of birth?
How do they verify that whether a person has filled it or not? Through my login ID or name or dob ?The reports I have read from knowledgeable people suggest the individual key hash they generate for you is calculated from partial name, DOB and partial address. The DOB remains in clear text in the final hash (don't know why, perhaps to allow a human to easily compare source and result?).
The implication that accidentally misspelling your name or DOB would produce a hash that would then be unable to be matched against data sets sourced from elsewhere, such as database of taxpayers or electoral roll, or Centrelink.So the hacker would need to have your name, address and date of birth from another source, but if they then had the ABS data set, they could re-identify your complete census response and match it to you. I believe the hashing algorithm is either understood, or capable of being brute forced if you have the data set and some name/address/DOBs.
It's not ideal, but would throw sand in the gears of automatic matching by the government and likely defeat all but the most determined hacker, who could probably get your personal details more easily other ways.
individual key hash they generate for you is calculated from partial name, DOB and partial address.
It's based on household, not person; I'm going to need links to these "reports".
One source is this SMH piece from privacy researcher Anna Johnston at consultancy Salinger privacy:
http://www.smh.com.au/comment/why-i-wont-be-filling-in-the-c…Example quote:
"Although there are certainly heightened privacy and security risks of accidental loss or malicious misuse with storing names and addresses, the deliberate privacy invasion starts with the use of that data to create a Statistical Linkage Key (SLK) for each individual, to use in linking data from other sources.Please don't believe that SLKs offer anonymity. SLKs are easy to generate, with the same standard used across multiple datasets. For example, Malcolm Turnbull would be known by the SLK URBAL241019541 in the type of datasets the ABS wants to match Census data against, including mental health services (yes, mental health!) and other health records, disability services records, early childhood records, community services records, as well as data about housing assistance and homelessness."
Apologies for the vague 'reports' citation in my original post, I was repeating from a message to a privacy mailing list, not a public post I could easily point to as a reference.
@fruit:
No, the log in code will be discarded once the census is complete. It is just a token to stop people submitting more than one response per address. Each individual in a household response will have SLKs produced personal to them.
Provide wrong DOB
Those providing misleading statements or information face a whopping fine of up to $1800.
sourceheard they will give you once chance to make it right - I can be wrong.
IANAL but it seems to me that the statistician needs to direct you to do so first, only then if you refuse can you get in trouble.
dob can be matched with your exact name in myGov, so not that hard to identify the error.
@blehgg: However you may only use Age if DOB is not known. Not likely to be caught though and they probably don't care.
they are lucky to have anything filled out ~
imagine the uproar if they fined anyone after the latest debacle
The worst part is that you may never know if your information has been hacked.
In Australia companies or the ABS have no legal obligation or requirement to inform the privacy commissioner or affected people of the data breach.
Trust is a 1 way street for them…Hey, what could go wrong?
Its not like we historically locked up certain nationalities in this country (sorry Japs, Italians and Germans) within my Dad's lifetime, or have seen examples of Census data being used in a similar way in the current day (US Census data on Muslim respondents sent to the NSA).But it is different now, its not like a nutter like Trump could ever be a possible leader, on institutions like the EU will remain stable to guard against any possible future threat.
In Europe Hitler used the data from the Census in WWII to round up the Jews and send them to the concentration camps.
In USA they used the data from the Census to arrest all Japanese americans after Pearl Harbour.
I wonder if the population was told that Census was going to be used to give them better infrastructures back then.
I am not implying that the Australian goverment will ever do anything like that, but if it is true that the data will be used for statistics and to plan government services, why do they need Names, Surnames, day/month/year dob and exact street address for that?
What are they not telling people?
I think it is a fair question.What are they not telling people?
Not to obey Godwin's law.
@Baysew:
Never heard that one…had to google it.
However it is a true fact…it really happened…you can google it ;-).
I came home one night 2 weeks ago & found the actual census form on my door step delivered just like in the past , filled it out Saturday night posted it this morning done .I wanted to ask did anyone else get the forms hand delivered ?
I'm doing family history research and just love the uk census data available on line (1901) which gives names, ages, addresses, relationships, occupations etc.
It's a pity that there is not a similar Australian resource.There has been a box to tick to allow your details to be released like that in 100 years for the last few censuses.
The first AU census was 1911. I wouldn't be surprised if they release that data in a decade or two when there is nobody left alive who personally knew anybody who has their details in the records. Assuming individual records were kept, even if they were stored without names attached (I don't know for the older census) I think you will find re-identification quite easy. For example, I know my Great grandfather's birthdate, and that he was a baker in Rockhampton in the 1940s. The data set for that postcode could easily be searched to show all men with that birthdate, so working out which of the handful of results was his would be pretty easy. You can obviously then go forward and backward across years with addresses etc.
They sometimes dont have the capacity to handle e-tax which is only a submission at the end. Much lower strain on the servers. Did they really think they could handle 10M or so submissions in the space of 10 hours.
easy with the right servers and back end systems
heck I rolled out Wifi Routers in Suncorp Stadium that handle a FULL stadium no worries
Surely the government can do betterSurely the government can do better
Exactly, just look at the stellar job they've done on the NBN rollout…
What's the bet the same thing happens tonight with everyone giving up last night?
It'll likely have more than 1mil transactions again during peak time.
You should make a poll of who failed to complete it last night, and will try again tonight.
It will be 'hackers' again, I guess.
Tweet: Census website down again. ABS blames it on 'Ozbargain Professionals'.
Gerry Harvey actively tracking down your SLK info now for infringing on his ™.
They made it all sound like you have to compete it on Aug 9th.
But it crashed for a lot of people.
Now their front page shows this:"The Census website is unavailable
We apologise for the inconvenience. There will be no fines for completing the Census after August 9. We will keep you updated."Yeah, you say you want me to fill it out and it's unavailable.
Then you say fill it out later, but won't even tell me when I could do that.
At least allow people to be notified by email when its available again.
And lastly you say fill it out later, but you don't even tell me what the last day to do that is.
A holes.September 23rd is the last day to have it in by.
They made it all sound like you have to compete it on Aug 9th.
Was it okay to complete it BEFORE Aug 9th?
Yes
Uh because this directly affects all Australians and it's compulsory!
What's your point?? So does a combination of what I mentioned above.
Give me anyone who's 'concerned' about their privacy and I'll easily come up with similar info on them with some govt dept or business. Not to mention the tons of potentially more invasive data that's available on 'BIG DATA'. People just don't have a clue. I'm all for privacy but that ship sailed very long time ago.
Hey gimme, did you get the job working PR for the ABS?
Completely avoiding the legitimate issues with hand waving, get a grip, nothing to see here etc.
If Facebook failed like this, plenty of people would be bagging them and plenty more would be saying you only have yourself to blame for trusting Zuckerburg.
We have no option but to trust the ABS, as they can force you to participate via fines. So we should expect a greater level of professionalism.I don't give a cr*p about the ABS or the census for that matter. I'm just amused at how clueless people are about their privacy and how big a deal they've made about completing a stupid 20mins form. Noone's going to get fined over technical difficulties. Minor inconvenience but sadly most people lack perspective in this VERY LUCKY country. Anyway, enough said…. good luck to all with their 'outrage' - Plenty of juicy material for the current affair shows.
"Hey gimme, did you get the job working PR for the ABS?"
No, on this forum, that would seem to be inherentchoice's job.
I don't know why some people waited until last night.
Because you're supposed to answer based on 9 August 2016 and some people were working during the day.
You mean people in your household transformed over the past week or 2?? lol. The only potential issue is accounting for 'guests' which lets face it on a Tuesday night it would be very small and either way not including guests is not a crime - they can simply include themselves on their own form.
yes its supposed to be a snapshot and if everyone did it a week early it wouldn't be an accurate snapshot of a single day
My wife's pregnant, due this month, so yes my circumstances could've changed by 9th August. Similarly someone in a household could've died on the 8th August. I guarantee that it happened, an average of over 400 Australians die each day. It's meant to be a snapshot of the 9th August 2016, not a snapshot of "some time around the end of July, early August", so why wouldn't people wait until 9th August to fill it out?…
yes yes we could all be run over by a bus. Geez some people really complicate their lives with matters that in the scheme of things makes little difference. A snapshot of a week or a snapshot of a day makes no meaningful difference. Anyway I'm over this. I've spent more time crapping on about it on a forum than what it took me to complete the census. I'm just glad my life is so much better than all the whingers out there.
New Census question - Are you really bothered by completing a census - YES - Move to a third world country and gain some perspective
Q2 - Are you really worried that your privacy will be compromised - YES - The Census should be the least if your worries when it comes to privacy. REALLY. Your privacy was lost long time ago.PS: Keep the negs coming. I enjoy being in the minority. Being in the majority you're just like everyone else - mindless consumers floating through life.
Are you really bothered by completing a census on or after the intended date? Yes you are, so do us all a favour and move to a third world country and gain some perspective.
It would probably have been fine if they hadn't kept going on about doing it August 9. I double checked the paper with the login and not anywhere did it say you could complete it at a different day. It just kept implying it had to be done on the 9th.
Their servers would never be able to handle everyone doing it on the same day, and I'm sure they must have known that, so I have no idea why they didn't more clearly state when you were supposed to complete it. Would have eased up pressure significantly and it would probably have been a non event. Oh well, at least Turnbull managed to complete it and inform us what a breeze it was. Such an ass.
Its a snapshot of Australia….. everyone has to do it on the same day
Actually, everyone don't have to fill it in on the same day. You have until September something.
Which we only found out late last night since it's not mentioned in any of the papers. You just have to fill it in as of 9th of August. Had they advertised this, we wouldn't have kept trying yesterday and just left it for another day. And servers would probably have been fine either way because they were obviously never going to be able to handle the peak hour load when people finish work if everyone is trying to do it on the same day.
We had it done at lunch time, had no problem logging in at that time :p
That's what the hackers said…
I think this article summarise the issue pretty well
http://eftm.com.au/2016/08/census-2016-the-10-million-online…
Obviously the load test of 1m concurrent users is not good enough.
"1,000,000 form submissions per hour was exactly what the site was tested for. The above message from the ABS assumes they expected 500,000 submissions per hour."
gee someone thought australia is just as big as nauru island.
Dont blame me I voted labor
Blame everyone who drew male genitalia on their voting forms…clearly that artwork was seen as a mandate to return a total dick to government.
Blame the donkey voters
I'd actually vote for a donkey before Talkbull…
If we elected One Nation, none of this would've happened
…cos' everyone would've just DDOSed Sensis instead of Census.
Learning from experience where I saw long queue during the day of the last election, I tried to fill in the census online before the 9th, just to see if they allow you to fill it in earlier - the same like how they allow early voting.
And it did allow me to finish, got the completion code etc.Lesson learned. Try to do things like this early (I also did early voting just to avoid the super long queue).
Just tried at 6.20 this morning. How shite is the site if it cant handle likely much less traffic than last night