It's 2021, which survey method should I use?

A telephone based approach

While telephone surveying still appears to be the most commonly employed approach when councils are administering surveys to represent the views of their broader community, this method is probably the one that has been most adversely impacted by digitisation.

Over the past ten years, the share of households with a fixed line telephone has dropped considerably and skewed older, with recent data from ACMA now showing that 60% of Australian homes are ‘mobile only’. The growth trajectory of mobile only households has been immense – doubling in the past handful of years – and shows no signs of slowing. While social researchers have done some really good work to modernise this methodology by integrating fixed line and mobile telephone samples, the increase in nuisance calls and other factors (e.g. automated call forwarding technology from unknown numbers in newer smartphones) has determined that survey participation rates using a telephone methodology has dropped to its lowest point ever according to Pew, the respected international research organisation. Beyond these challenges, a telephone methodology needs interviewers to conduct the survey, making it clearly the most expensive and slowest of the approaches discussed in this post. Taking all this into consideration, our recommendation to local government professionals is to really question whether this once appropriate methodology is still the best way to generate a representative sample of your residents at scale, and for an acceptable price, even if you have been using it for many years.

Social media advertising

The vast majority of Australians are using some form of social media, and that number is growing every year. According to industry statistics, 20.5 million (80%) Australians accessed social media in January 2021 and approximately one-third of our entire time online was spent engaging with social media.

Just as the absolute number of Australians available on social media is increasing and the relative share of our online time is being spent more and more on social media platforms, there is a growing body of international research exploring how well social media advertising performs for surveying purposes. .id has been at the leading edge of using social media advertising as a way to help our local government clients better represent their community’s views using surveys since 2019 and, our trials and client work using this approach continue to reinforce our belief in the benefits of this method. In 2019, we tested our new Living in Place survey in two very different LGAs (Blacktown in Sydney’s west and Kingston in Melbourne’s south) using a social media advertising approach to see how well the returned surveys reflected the characteristics of both populations. After surveying around 500 people from each LGA, our analysis showed that the residents’ age, gender, language spoken and home ownership profiles from the survey largely reflected the population across both communities. We were also happy with how well the samples reflected the distribution of suburbs within each LGA spatially, and even ran a test to recalculate each LGAs SEIFA. In both tests, the SEIFA derived from the survey closely reflected those reported in the 2016 ABS Census. We did this to understand the inclusivity of social media advertising for our local government clients. After all, councils are there to represent the needs of all – and not some – of their residents.

While it is true that older and indigenous Australians are currently less well represented online, the digital divide is closing. And, with the federal government, private sector and advocacy groups working together to reduce that divide even further, we are confident it will continue to decrease over-time. Beyond the hard statistics and government policy, there are also advocacy groups who represent the interests of senior Australians who participate in this conversation. In one recent publication, National Seniors Australia reported that the number of people aged 75+ who use social media doubled from 19% in 2019 to 42% in 2021 – busting the myth that older Australians are not online, or on social media.

While one of social media advertisings key features is its ability to be very specific with geographic and demographic targeting, running a campaign is not as simple as whacking an ad up on the socials and watching the completed surveys roll in. We work with a leading social media agency to design and administer the social media campaigns for our Living in Place clients, build trust with the community by creating relatable and empathetic advertising communications and use a data-led approach to ensure that our survey participants’ demographic and geographic characteristics largely reflect the composition of our client’s LGA. Since launching Living in Place in 2020, we’ve been able to deliver large and credible samples for a diverse set of LGAs – ranging from the Sunshine Coast in Queensland to Strathfield in suburban Sydney, to the fast-growing City of Ipswich on Brisbane’s fringe and even the Huon Valley in southernmost Tasmania.

Social media advertising’s ability to deliver credible samples that reflect the geographic and demographic characteristics of our clients’ LGAs, and the volume of surveys it can deliver for our local government clients compared to available alternatives for a reasonable price, determine that it is our survey research methodology of choice and one that we believe councils should seek to learn more about and consider if they are looking for a modern, efficient and proven way to best represent the views of their entire community in local government decision making.

List based surveying

List based surveying – sometimes referred to as convenience surveying – is a methodology used when only those who exist on a list accessible to an organisation are invited to participate in a survey. List based surveying is convenient, fast and often employed exclusively in a digital environment using CRM infrastructure. We believe list-based surveying approaches can be really beneficial for councils when they are seeking to understand experiences of services delivered through, or supported by, local government professionals. Measuring the experiences of people who use maternal child health services, libraries, aquatic and gym facilities, and the like are great examples where list-based surveying is appropriate. And, given its ability to trigger survey requests directly after experiences, this approach can be prove very helpful for local government professionals seeking to identify and remedy issues related to service delivery.

While list based surveying can be very impactful, we believe it is less appropriate when councils are seeking to inform, monitor or evaluate policies that affect the entirety of its community. Surveys used to bring a resident lens to community visions, council plans, state and federal government grants and other advocacy should aim to be much broader in their reach – and use a methodology that can efficiently capture the views of more residents who have a passing interest in council policy, or even benign relationship with their LGA. Using list-based surveying – or even an exclusively opt-in approach on a council website (e.g. a have you say type site) – for this purpose runs the risk of skewing survey results towards the perspectives of a few loud voices who may take a great amount interest in council’s business but not necessarily have an equally high-stake in the outcome of the LGA’s policy or advocacy goals.

Online panels

Broadly speaking, online panels are created by digital companies that source, grow and maintain lists of people who are willing to participate in online surveys (e.g. via email, sms) in return for some form of incentive (e.g. cash, points towards goods, entry into prize drawers). There are a host of online panel providers in Australia and, while it started as a cheaper and faster alternative to telephone surveying, it has become a commonly accepted methodology in recent years. Many large ongoing federal government surveys, for example, use online panels as their sample source.

Launching surveys and receiving results using an online panel is typically quick and relatively inexpensive. While a great option to represent the views of the nation, other very large geographies and people who fit a particular demographic or employ a behaviour of interest to the researcher (irrespective as to where they live) online panels are probably not the best option for councils to represent their community’s perspectives. We believe this for two reasons, firstly because the relative share of the community available to be surveyed using an online panel approach within any given LGA is likely to be very small in absolute terms, and, secondly, those who are available may be skewed spatially within the LGA, by demography and life-stage. Unless you are one of the few very large councils we would recommend not considering a panel based approach to represent the views of your broad community.

In summary, when reflecting on the pros and cons discussed across these four survey methodologies it’s important to remember that no one approach will be perfect. When choosing which (if any) is best for your council, we would encourage you to have a sound understand as to which members of your community will be less represented as a result. Having this understanding, and being transparent about your chosen method’s limitations, should leave you well positioned to find other (perhaps non-survey based) ways to integrate the views of these resident groups to help inform your big picture decision making.

Links to more information…