"Big Data Is Not the Answer to Everything":Advertising Practitioners' Perception of Big Data
Big data is a popular buzzword in many industries today. As more data about consumers and citizens become available due to ever advancing digital technologies, learning how to effectively collect, sort, interpret, and use big data becomes an urgent task and capability for many companies. However, big data is still a group of numbers that professionals must manipulate in order to make sense of it all, which then can be used for particular purposes. This study sought to examine the ramifications of the explosion of big data in advertising through interviews with eleven decision makers in the advertising industry. Four themes emerged from these interviews: Great(er) Expectations, Overworked and Insecure, Humans vs. Machines, and What is "Big Data" Anyway? Implications of these experiences were examined. Through in-depth interviews with professionals managing big data every day, results contribute to the advertising literature by bringing a greater understanding of big data's implications and possible societal and cultural impacts.
advertising industry, big data, interviews, qualitative research, technological affordances
Microsoft tycoon Bill Gates once wrote, "We always overestimate the change that will occur in the next two years and underestimate the change that will occur in the next ten" (Gates 1996, 316). In this quote and throughout his book, Gates suggested it is nearly impossible to predict growth in society, and this is especially true in understanding the impact of technology. Even so, it is important to recognize that growth and advancement will always happen. Arguably, some of the most misunderstood growth in the advertising realm is in the area of collection and usage of large sets of data now available to practitioners, popularly known as "big data." Big data has come to include social media metrics, online purchase behaviors, location-tracking applications, along with several other types of consumer personal information collected and sold by various companies.
Gates was not wrong when he said we cannot predict what is in store for the next 10 years, and technological progress in the advertising profession has not slowed down. Programmatic advertising, or the use of software to purchase and place advertising automatically, makes up its own multi-billion dollar industry within the strategic targeting sector (Fisher 2018). Artificial intelligence (AI) is entering our homes through products like Google Home and Amazon Echo, where tech companies can collect minute-by-minute consumption data in order to tailor its offerings to an individual's needs, preferences, and behaviors. As technology advances, so too do the methods that advertisers and marketers implement in the quest for more information on consumers and their behaviors (Kitchin 2014). Society's increased dependency on the Internet and compatible platforms makes information more accessible, as millions of individuals are using multiple applications, devices, and websites every day.
The focus in marketing communications has switched from "how do we find out about consumers," to "which pieces of information about consumers are most important." Big data has been a large contributor to this transformation, as advertisers and marketers can use these large data sets to create more effective campaigns because they offer greater insight into current and potential consumers (boyd and Crawford 2012; Kim 2014; Shaw 2014). With all of the available data, learning how to most successfully collect, sort, interpret, and use big data becomes an urgent task and must-have ability for many companies and organizations, especially in the advertising industry.
Over the last decade, research around big data and its uses, applications, and concerns has increased, indicating the importance of the topic in multiple disciplines (Biesdorf, Court, and Willmott 2013; Rappaport 2014). Much of the research published in this area is laid out like extended literature reviews, with the aim of addressing future research agendas and creating new frameworks of knowledge (Chen, Chiang, and Storey 2012; Holtzhausen 2016; Marht and Scharkow 2013; Newell and Marabelli 2015; Sivarajah et al. 2017; Yin and Kaynak 2015). Frequently covered topics have included ethical concerns, challenges, and benefits faced by practitioners; the growing importance of big data in industry applications; and general implications of its increased popularity. Big data seems to be understood from the academic perspective, but it is important to explore these ideas from the perspectives of practitioners, as well. Being able to apply previous academic thought directly to advertising and marketing practitioners' experiences can support both the expansion of this topic within these academic realms and potential implications for the industry.
Despite the reliance on big data in advertising, there are concerns around the definition and use of the information big data provides. Kitchin (2013, 2014) listed seven characteristics to consider when talking about big data: huge volume, high velocity, diverse in variety, exhaustive in scope, indexical, relational, and flexible.1 As with most concepts, big data is defined differently depending on the context (boyd and Crawford 2012; Fulgoni 2013; Goes 2014; Kim 2014; Kitchin 2013, 2014). To highlight some differences, boyd and Crawford (2012) argued that the market is one of the biggest influencers of the application and usage of big data, specifically among for-profit industries. One definition of big data cannot possibly encompass all meanings, as it differs for each sector, as described by the authors: "The market sees Big Data as pure opportunity: marketers use it to target advertising, insurance providers use it to optimize their offerings, and Wall Street bankers use it to read the market" (663).
The goal of this article is to narrow down what it means to use big data in advertising from the perspective of practitioners. Questions also arise around the invasion of the privacy of users of the monitored platforms collecting data, and whether or not current collection methods are ethically justified (boyd and Crawford 2012; Mahrt and Scharkow 2013; Yang and Kang 2015). It is essential then to examine these perspectives from those who actually benefit from the influx of data. Through the use of qualitative interviews, this article describes the current experiences some practitioners in the advertising and marketing fields have had with big data, specifically addressing the benefits and disadvantages of the industry's reliance on these relatively unstructured data sets.
What Is Big Data?
Because the concept of big data has been applied in various disciplines and fields, different scholars from diverse backgrounds have defined it from different perspectives. For example, Fulgoni (2013) described big data as a by-product of the use of computers to solve an operational problem. Goes (2014) defined big data as a massive amount of observational data of different types used to support different decisions within decision time frames. Similarly, Kim (2014) refers to big data as the data sets and the related analytical techniques of storage, analysis, business management, and visualization.
boyd and Crawford (2012) warned that big data cannot be defined simply by its size; rather, it is important to look at its relationality to other data. They further indicated that big data is not just a very large data set with particular tools and procedures used to manipulate and analyze them. Rather, big data might change the meanings of learning about knowledge and bring new possibilities and limitations in the areas of multiple disciplines. Built on this understanding, boyd and Crawford defined big data as a cultural, technological, and scholarly phenomenon that rests on the interplay of technology, analysis, and mythology. On top of Kitchin's (2014) seven characteristics mentioned earlier, he also commented on the implications of the exploratory nature of big data: ". . . others suggest big data ushers in a new era of empiricism, wherein the volume of data, accompanied by techniques that can reveal their inherent truth, enables data to speak for themselves free of theory" (3).
The most recent attempt at defining the term big data came from Wiesenberg, Zerfass, and Moreno (2017), who describe it in the context of strategic communications, specifically public relations: "Big data denote huge volumes and streams of different forms of data from diverse internal and external sources and their constant processing" (2). This definition features three important parts through 1) huge volumes and streams, 2) different forms of data, and 3) constant processing. These three parts stem from Laney's (2001) "Three V's" approach to big data, which mentions volume, variety, and velocity to distinguish big data from "small data." While many of the big data articles in communication have focused on marketing communications due to its important implications for an industry that aims to understand as much as possible about consumers, there is room for further exploration of this concept at the qualitative level. Jobs, Gilfoil, and Aukers (2016) used semi-structured interviews to examine the spending transformations of marketers, using the information to talk more about how budgets are being spent. Other research has chosen to focus more on definitions and descriptions of big data in an attempt to simplify the confusing nature of big data (Bolin and Schwarz 2015; boyd and Crawford 2012; Harper 2016; Kitchin 2013, 2014; Marht and Scharkow 2013). Reaching out to practitioners in an attempt to directly interact with and explore the phenomenon of big data in the everyday work of marketing and advertising can further scholarly efforts in empirically studying the concept. As mentioned previously, the definition of big data changes when the context changes. Therefore, the investigation of practitioners' perception of big data. which reveals the core meanings and connotations of this concept in the unique context of the advertising industry, would enrich our understanding of the dimensions, nuances, and implications of big data in the real world.
History and Evolution of Data
To really understand how big data infiltrated the advertising industry, there must first be an understanding around the history of data as a tool for organizations. The collection of "data" of individuals has been a method for learning about the population as long as the United States has been a country, but it was the need for mechanization, then later automation, that has helped larger data sets find their way into society. Regarding the need for mechanization, at the end of the 18th century, the United States experienced a population boom in which manual tabulation and storage of Census data became a burden for those who were in charge of this task, and nearly impossible to complete in the 10 years between collection periods. To solve this problem, Herman Hollerith dramatically improved (and arguably saved) the Census process through a punch-card type of machine that "reduced a ten-year job to three months" (da Cruz 2019; Truesdell 1965).
Since Hollerith's initiative toward mechanization, data collection, storage, and ultimately usefulness have only become faster and easier. At first, these processes were most relevant for government documents and records of academic findings, but eventually retrieval and strategic use of individual information become more important to companies and advertisers specifically. At the beginning of the 20th century when social science methodologies formalized, advertisers took a turn toward "scientific advertising" where they gradually used surveys to help them to better understand a large group of consumers. A few decades later, during the economic boom following World War II, the production of radio and print advertisements increased and was integral in the success of the mass communication and media industry, especially television (Kierlanczyk 2016). Research done in this time that specifically focused on consumer behavior and preferences helped academics and businesses alike learn about the psychology behind product desires and purchase decisions. Market research evolved from learning about the masses through early survey-style methods to learning about the individual through focus groups and ethnographies. This information, combined with more quantitative business records like sales and overall financial performance, revolutionized and shaped the way advertisers created copy and how creatives directed their messages to segments of consumers, rather than "the masses" (i.e. the general population). Advertisers' integration of quantitative and qualitative data collection methods was widely used until the birth of the computer and the Internet which brought the technological possibilities of data automation and advanced data analytics.
The Internet boom in the 1990s jumpstarted the use of more complex analyses to discover patterns of mass consumption, relying more on personal information collected away from traditional market research tactics that focused more on group profile (Kierlanczyk 2016; Winshuttle 2019). At this same time, the term "business intelligence" popped up more frequently in business settings to emphasize the intelligence associated with machine-driven analyses, made possible by the coding capabilities of computers. These strategies were used to bolster and promote data-driven decision-making, including how to better target segments of the population through messaging. Essentially, big data inevitably entered the advertising realm through the combination of computer science, psychology, and strategic communication to better understand what consumers were paying attention to through media consumption and why. As the population grew and technology continued to advance, more people signed onto the Internet on more devices, allowing for the collection and aggregation of larger and larger data sets. Naturally, businesses took advantage of all of this information to learn as much as possible about their current and potential customers, and incorporating big data has now become standard in many advertising practices.
Big Data and Marketing Communications
As mentioned previously, big data is still a confusing term within advertising and marketing. This ambiguity in meaning makes it difficult for marketing researchers, in both the industry and academia, to find an appropriate use for the data. Along with the differences in technical definition, practitioners are faced with ethical dilemmas when choosing to collect and analyze big data (Bollier 2010; boyd and Crawford 2012; Mahrt and Scharkow 2013; Metcalf and Crawford 2016; Nunan and Di Domenico 2013; Yang and Kang 2015; Zook et al. 2017; Zwitter 2014). As advertisers are dealing with the collection of personal data of consumers, one of the biggest issues is the consent of obtaining such information ethically. Cookies and user agreement documents (e.g. "Terms and Conditions") are some of the most important means to receiving immediate consent from Internet browsers, but how many people actually understand what is being sold and traded about them through their use of digital platforms? Those in the business of collecting and selling data are well-versed in the implications of such activity. However, who is in the business of enlightening consumers? Critical and satirical shows like Comedy Central's South Park and Netflix's Black Mirror have tried to expose the problems. South Park lampooned "Terms and Conditions" documents (among several other marketing, advertising, and big data criticisms) and Black Mirror warned against the exploitation of personal data such as credit scores in everyday interactions. Are the appropriate conversations being had among advertisers on how best to handle these potential, sometimes already developed, issues? Although big data appears to solve many problems that have plagued the communications field for decades, there are many complications researchers are facing when attempting to use big data in the study of populations. Namely, most problems center on the relative inexperience with large data sets in some social science-based fields like advertising and marketing.
Despite this difficulty, research has attempted to establish strategies for marketers and advertisers to use big data in the ever evolving new media environment (Bollier 2010; Culotta and Cutler 2016; Fulgoni 2013; Fulgoni and Lipsman 2014; Jayaram, Manrai, and Manrai 2015; Kim 2014; Knudsen and Kjeldgaard 2014; Liu, Singh, and Srinivasan 2016; Moretti and Tuan 2014; Trusov, Ma, and Jamal 2016). Bollier (2010) indicated that the most important application of big data in the marketing field is the use of real-time data correlations to drive business decisions. For example, geo-location data discovered that regional consumer demand could be correlated with "the length of time that consumers are willing to travel to shopping malls" in a specific region (Bollier 2010, 2). A more famous example involved a popular snack product, Pop-Tarts. After examining data on purchase behaviors before a hurricane in the US South, international retailer Wal-Mart discovered "that strawberry Pop-Tarts increase in sales, like seven times their normal sales rate, ahead of a hurricane" (Hays 2004). Understanding these specific behaviors before the era of big data was virtually impossible due in large part to the sheer volume of information that needs to be gathered to find these patterns. The possibilities seem limitless when data is used with a clear and specific goal in mind.
Fulgoni and Lipsman (2014) further provided detailed descriptions of changes and possibilities brought by big data, such as better incorporation of multiplatform data, availability of disaggregate consumer level data—data collected from several, unrelated, sources—and a much faster speed of delivering results. While acknowledging big data's positive impacts on marketing, Fulgoni (2013) also pointed out that when using real-time data to make decisions, marketers may use short-term marketing strategies that neglect the importance of building long-term brand equity. When this occurs, both marketers and consumers lose out, as brand equity is the differential value generated by a recognizable brand and product when compared to a non-name brand equivalent product. Essentially, when marketers focus on short-term outcomes using big data to attract more sales in a short period of time, the brand's emphasis on profits over relationships can produce negative long-term implications. For example, banner advertisements are a popular means to reach potentially millions of consumers. They are inexpensive and programmatic, which means that ad space is bought using an algorithm and data from users' previous online activity that is tracked through cookies. It can be difficult to determine where on the Internet a brand's advertisement will show up, though, which may lead consumers to believe a brand is associated with a certain type of content (e.g. Kellogg's ads showing up on the far-right American news website Breitbart). The general lack of awareness around the use of data on the Internet continues to be a challenge for brands and advertisers (Tedford 2017).
Along with understanding the benefits and challenges of big data within marketing communications, scholars have also worked to build effective conceptual frameworks to understand the nature of big data. Jayaram, Manrai, and Manrai (2015) paired 10 present-day technologies—digital profiling, segmentation, websites, search engine marketing, campaign management, content management, social media, mobile applications, digital collaboration, and analytics—with 10 corresponding market characteristics in the creation of three "Iditarod-style" models2 for high speed, medium speed, and low speed countries in Eastern Europe. These frameworks gave organizations a starting point to build strategic marketing campaigns around a target consumer's access to and knowledge of specific technologies. Moretti and Tuan (2014) explored the relationship between Relationship Marketing (RM)3 and the recent explosion of Social Media Marketing (SMM)4, determining that SMM can be considered a subset of RM. This "evolution" to the use of SMM is characterized by more transparency and engagement from brands in response to increased interaction and direct communication between consumers and organizations through social media platforms.
Culotta and Cutler (2016) investigated an automated method for inferring attribute-specific brand perception ratings by mining brands' social connections on Twitter. Using a set of over 200 brands and three perceptual attributes of eco-friendliness, luxury, and nutrition, they compared the method's automatic ratings estimates with directly elicited survey data, and found that the big data-based analytic tool of automated social network mining provides a reliable, flexible, and scalable method for monitoring brand perceptions. In a similar study, Liu, Singh, and Srinivasan (2016) analyzed a big set of data from five sources of Twitter, Google trends, Wikipedia pages, IMDB reviews, and Huffington Post news regarding TV show comments. They found that in contrast to basic surface-level measures such as the volume of comments or sentiments in Tweets, the information content of Tweets and their timeliness significantly improve forecasting accuracy. In addition, they found that Twitter is a better predictor of TV show demand than other online data such as Google Trends, Wikipedia views, IMDB reviews, and Huffington Post news. Focusing on user profiling, Trusov, Ma, and Jamal (2016) proposed and empirically tested a big data modeling approach that uncovers individual user profiles from online surfing data and allows online businesses to make profile predictions when limited information is available.
While more available data may seem like a dream come true to advertising and marketing departments, the evolution of the collection methods comes with some complications. Previous literature emphasizes the ethical complications of big data almost as often as the development of uses for the information (boyd and Crawford 2012; Mahrt and Scharkow 2013; Yang and Kang 2015). These considerations are important for practitioners because some consumers are becoming more aware of and familiar with organizations collecting data from the Internet and other digital platforms without direct knowledge of where the information is going. One of the biggest issues raised was the potential for misinterpretation of data due to the very large sample sizes (boyd and Crawford 2012; Mahrt and Scharkow 2013). These large sample sizes can invent a relationship between variables that simply is not there, as researchers are not yet familiar with the most efficient way to analyze complex data sets. For example, big data analysis once showed a strong but spurious correlation between the changes in the S&P 500 stock index and butter production in Bangladesh (boyd and Crawford 2012). While the correlation between those two variables may be significant, and analysis of the collected data sets seemingly support a relationship between them,5 there is need to go back to the adage that correlation does not mean causation.6 More data opens the door to more possibilities for misinterpretation, with stock index and butter production serving as just one example of how these mistakes occur when data is not examined within a specific context. boyd and Crawford (2012) went on to develop six themes that address potential ethical questions concerned with the collection and use of large data sets from the Internet and other digital platforms after recognizing some of the ethical issues encompassing the collection and selling of data for strategic marketing purposes. The themes focus on the definition of knowledge, objectivity and accuracy of data, size of data, context of data, accessibility of data, and digital divides.
Along with misinterpretations, some of the more important discussions mentioned ethical considerations for claims of "anonymity," and the chance for an "information divide" to be formed through unfair distribution of data sets by the organizations that own the data (Mahrt and Scharkow 2013). Data from the Internet is not free, as many technology groups benefit from the fact they can sell users' information to other companies. Giants like Facebook and Amazon make it very costly for non-tech giants to buy user data, and The Atlantic found these giants actually exchange user information between each other (Madrigal 2018). This leaves a lot of user information in the hands of a few. On top of this, much of this information was collected and distributed without user knowledge or agreement. With more data breaches each year, privacy is a big concern for consumers, but not for the companies who own the data (Zlatolas et al. 2015). Yang and Kang (2015) also found that culture plays an important role in a consumer's perception of their right to privacy on Facebook. This is why the use of big data by marketers should not be lumped into one large category. Ambiguity of the definition of big data and the care needed to use these data sets has led researchers to urge those in both business and academia to consider implementation of stricter standards on collection and analysis of big data for strategic purposes to avoid damaging consumers' trust (Marht and Scharkow 2013; Yang and Kang 2015). The questions that arise from the uncertainty around big data requires a thorough analysis of the term and its uses from the perspective of those who come in contact with big data every day. With this in mind, this study sets out to answer the following research question: What are advertising practitioners' current perceptions toward and experiences with big data?
This qualitative study used in-depth interviews with current advertising and marketing practitioners in the United States.7 By using the voice of these practitioners, this article explores the industry's perceptions toward big data, specifically the benefits and downsides that come with big data, as well as possible cultural and societal implications from the perspective of those practitioners.
The views of current practitioners are important in understanding the influence of big data on the industry, as well as the future implications of accessing large amounts of personal information from consumers. Previous research has attempted to explore the above two topics within big data, but this has not been achieved by utilizing the knowledge of those specifically using these complex data sets daily for strategic decision-making (boyd and Crawford 2012; Kim 2014).
This study focused on advertising and marketing practitioners in the US who were employed at companies using big data to make strategic decisions. In addition, participants were all middle- or upper-level associates in advertising or marketing roles at US companies with enough professional experience to understand broader industry operations and big data usage. While there are other groups more involved in collecting and analyzing big data, like statisticians and analysts, it was more important to discover the perceptions of those who must interpret the large and complex data sets. Knowing what the data is saying and how to use it to reach consumers is the ultimate goal of market research, but with the seemingly limitless amount of information available to agencies and firms, this can turn into a difficult task (boyd and Crawford 2012; Crawford, Miltner, and Gray 2014).
Purposive and snowball sampling techniques were used to recruit participants for this article. The requirements for participation were: 1) the individual must be a middle- to upper-level associate in an advertising or marketing role and 2) the individual must utilize big data in their strategic decision-making process. Participants were found through personal contacts and email recruitment. With digital advertising exploding in popularity, searching for agencies around the country was the best method to recruiting as many participants as possible. Specifically, we used Redbooks.com; the database on advertising agencies was used to find top advertising agencies and their contact information. Once emails were sent, other participants were recruited using snowballing methods in which the researcher asked the interviewees to recommend other participants. In total, 11 participants, identified with pseudonyms to ensure confidentiality, were recruited: Carter, President; Ben, Director of Planning; Craig, Analytics Strategist; Lucas, Chief Executive Officer; Liz, Development and Client Team Member; Randy, Director of Analytics; Jordan, President; Scott, Founder; Christine, Managing Director of Research, Insight, and Strategy; Kathy, Industry Consultant; and David, Educator and Industry Consultant.
Even with a small number of respondents, McCracken argues, "For many research projects, eight respondents will be perfectly sufficient" (1988, 17). Qualitative research is about examining the culture of a purposive sample of individuals, and phenomenology is even more so meant as an approach to discovering and appreciating the experiences of a specific, limited group of individuals (Creswell 2013). The criterion to determine the number of participants was guided by saturation, which is the point at which no new concepts and themes emerge (Corbin and Strauss 2008). Saturation was achieved with the ninth participant, but two more participants were recruited to test this saturation and see if there were any possible new insights.
In-depth interviews were used, which is a powerful research tool that "gives us the opportunity to step into the mind of another person, to see and experience the world as they do themselves" (McCracken 1988, 9). It sets only broad parameters for the discussion, leaving participants free to tell their own stories. After receiving an informed consent form, interested practitioners set up a date and time for a one-on-one telephone interview with one of the researchers. All interviews were audio recorded for analysis. The interviewer introduced the study to the participants at the beginning of the phone call, while also going over the informed consent procedures. Once participants agreed to the conditions of the study, they were asked general introductory questions about their background, which led into conversations about their perceptions toward big data. The interviewer used a semi-structured interview guide to allow for less restrictive conversation (see Appendix). All topics in the guide were covered throughout the interview. Each interview lasted around 30 minutes.
Interviews were then transcribed and used for analysis. In the first stage, one of the researchers immersed herself in the transcripts to obtain a general sense of the whole. Then, she started open coding to identify initial codes. In order to do so, she carefully read each single comment, looked for repeated words and similar terms, and grouped comments based on closeness of meanings. After that, she re-read all the transcripts and paid close attention to relationships and internal logics among the categories. By using the analytic induction method (Taylor, Hoy, and Haley 1996), she condensed the initial codes into four potential themes. With ongoing analyzing and reviewing, four themes were refined and confirmed, which will be discussed in detail below: Great(er) Expectations, Overworked and Insecure, Humans vs. Machines, and What is 'Big Data' Anyway?
Two measures were used to ensure the quality of the analysis and validity of the emergent themes: 1) peer review, during which a qualitative researcher who is familiar with the topic analyzed the data and reviewed the emergent themes; and, 2) external audit, during which a qualitative researcher who is not familiar with the topic examined the process and the product of the account accessing accuracy (Creswell 2013).
After interviewing 11 participants, several themes emerged about practitioners' experiences with big data. Overall, those in advertising feel that the influx of and reliance on big data is beneficial for data-driven decision-making, but there are major challenges because there is still very little organization and understanding around how best to utilize these massive data sets. Notably, there are expectations that agencies will find a way to incorporate big data into strategic campaigns, even if they do not have the proper resources (e.g. data scientists on staff) to actually conduct and then interpret and apply the findings. In all, there were four main themes that emerged from interviews with those who are familiar with big data in advertising. Each of these themes will be discussed in turn.
Theme #1: Great(er) Expectations
Naturally, advancement in technology can change and enhance the functions within an organization and even an entire industry. To practitioners, big data has forever changed the way the industry functions due in large part to clients' increased expectations surrounding results. Practitioners are expected to deliver all of the answers with the increased access to information on customers. No matter the size of the organization, or the type of clients served, participants recognized the need to utilize big data because clients believe their money is best spent when big data is involved. This idea stems from the seemingly endless sources of information available about consumers, specifically related to their behaviors on digital Internet-enabled devices. Carter, an agency president, noted how access to this information is "changing the entire industry": "I think it's making the entire industry more accountable, more measurable, more targetable, more accountable to return on investment. I think it's great for brands because they get to be more informed, intelligent with data-centric media buys."
As Carter indicates, marketers seem to greatly benefit from big data by obtaining more information and data in order to make more accurate and improved strategic decisions on advertising such as budgeting and measurement of effectiveness. In fact, participants are beginning to redefine advertising around the potential applications of big data. Craig believes "the nature of what really is advertising" now includes "media to creative to making websites to social media." Agencies in the digital realm can no longer focus only on selling and buying ad space. Rather, as Craig, an analytic strategist notes, they must embrace the hype that is big data because "the more frequently people (clients) get used to seeing data, the more frequently they want it."
Practitioners look forward to the potential influence of big data on decision-making in better serving their clients, while also recognizing that their responsibilities are constantly evolving and likely increasing as clients are becoming more aware of the availability of information. Lucas, a CEO, admits that this is "exciting," but also a "burden": ". . . people are going to be expected to know way more about the data than they were expected to know in the past. Even the creative person, who will now have to understand the outcomes that it's driving and how we're tracking it, etc."
Thus, Carter, Craig, and Lucas suggest that data literacy has become a necessity for practitioners in today's big data era. Sometimes big data is not the right answer, though, and it becomes an extra task for practitioners to convince clients that data cannot tell a consumer's entire story. As an analyst, Ben, a director of planning, looks at numbers all day, but noted that sometimes his agency decides it is best to step away from these data and say, "We need to try things that may not look like they are going to work from a data perspective, or from a testing perspective, but we want to take a chance on this because we really do think it will work."
The non-data perspective8 is critically important in today's data driven world. In particular, in a creative industry like advertising, strategic decisions cannot solely depend on numbers. Furthering this notion, Ben noted that while data allow him to "hone in on the human truth or human insight," there are many ways to reach this end goal that likely includes "primary research and literally getting out and talking to people." Liz, a development and client team member, said utilizing big data in her company meant they were combining "qualitative and quantitative; we're looking at big groups of people, big ideas." Ultimately, there was a consensus around the idea that big data equals more of everything, which leads to more responsibility for every individual in an agency.
This responsibility that comes with the need to learn how to best handle big data is actually reshaping the culture of advertising, as organizations are shifting from the silo structures of the past to more collective efforts. In the past, researchers conducted research, writers wrote copy, and advertisers did not cross over into marketing or public relations. All of these responsibilities now blend in the digital age (Zajack 2018). More minds are needed to collect, interpret, and implement strategies formed by these large data sets because client expectations are high, but practitioners' technical knowledge surrounding big data is relatively low. This shift was described by Craig, an analytics strategist:
It also is a big thing for us that people who weren't used to owning the data understand that it is their data and that it's all our data. That we are here to make decisions together and that analyst will be a part of that conversation, but that it should be a collaboration. . . . It means we have to be siloed, but also work together and be collaborative. This makes it much less clear which individual is contributing what, but I think the end product ends up being better so everybody will end up feeling good about it.
Here Craig highlights the organizational shift that is occurring with the use of big data, bringing both negative and positive implications. With this, practitioners are both excited about the evolution of the industry, and are feeling increased pressure on their organizations to use these new technologies, whether it is the best move or not. Because of this pressure, practitioners sometimes feel overwhelmed about big data. In particular, they feel impotent on both accessibility and applications of big data, which construct a sense of being overworked and insecure, the second theme that emerged from our interviews.
Theme #2: Overworked and Insecure
Expectations are higher, but practitioners are not so sure they can keep up with the demand. Access to information and technology is only one aspect of success with big data. The other major task involves actually knowing how to use all of the data available. In the industry, large tech companies such as Google, Amazon, and Facebook have led the way in applying and using big data, but most participants were not members of these organizations. Those big tech companies have significant financial and human resources to acquire, collect, analyze, and export, which sets the standards for big data applications in the industry. For those practitioners we interviewed who worked at smaller organizations, they felt they were always trying to keep up with those who set the trends for big data usage. Randy, a director of analytics, believed big data management is characterized by specialized infrastructure, like Apache Hadoop, a collection of open-source software utilities that facilitate using a network of many computers to solve problems involving massive amounts of data and computation. However, as he noted, "we don't have that type of infrastructure."
To Scott, a founder, it was obvious that those "who built their business on the Internet and on data" would have the upper hand in the industry. Data is accessible, but it is not going to analyze itself, and practitioners noted that recruiting highly-skilled individuals who were trained to work with big data analytics were much less reachable in the advertising industry. Ben, a director of planning, talked about his personal challenge in this area: "I think today there is so much out there that we just have to stay informed. For example, for me, we do our best to try to stay on top of everything, but because I am trying to do so many different things for clients and things related to the business, I don't have time necessarily to keep on top of what's out there."
The first theme of great(er) expectations, and the second theme of overworked and insecure, build off of one another as practitioners want to be able to keep up with client expectations and industry standards, but this is a difficult task with limited talent, time, and technical resources. The large tech groups hoard the best talent due to their reputation of success, financial strength, and stability in the field, while the smaller agencies continue to rely on a few people who may or may not specialize in big data.9 More data leads to more complex data management. This complexity then requires practitioners to have knowledge of these systems. Hiring individuals who are skilled with these technologies would help the situation, but Jordan, an agency president, noted one of the biggest challenges for his company was finding the right people who had the experience to work with big data: "Our big thing is the people with experience don't have the time, and the people with time don't have the experience. You know, it's really true. Without the experience and knowing what questions to ask of data, the data is worthless. That's a challenge."
If the most talented individuals are picked up by a small group of powerful companies, while the advertising agencies need more advanced data scientists who can fill the current gaps in understanding data, this may become problematic for society's overall perceptions toward advertising as a whole. In the past several years, especially since the introduction of programmatic advertising, consumers' trust in advertising has decreased dramatically thanks in large part to "aggressive retargeting" tactics on the Internet (Roderick 2017). While the advertising revenues increase because of these tactics, the perceptions toward the industry as a whole are falling, leading consumers to implement tools like ad blockers. Really understanding what consumers want from data-driven advertising needs to be addressed, but has been slow to happen. A question that arises from the growing concentration of hiring among a few companies: What happens to the advertising field when Amazon, Facebook, and Google lead the digital charge? It is no secret the culture of the ad tech industry has transformed dramatically since these large tech groups have set the standards for everyone. Kathy, an industry consultant, agrees by commenting on the pressures to keep up with well-resourced tech giants: "The idea is that these are people who are students of consumer behavior or students of business behavior. . . . And then we're putting in this structure, this accountability, and then speeding everything up. Like, everything needs to be done by tomorrow, and we need to have the results by tomorrow, and we need to adjust everything by tomorrow."
The stricter time demands are especially challenging in industries working within the digital realm, and advertising agencies using big data conform to those standards. This is mainly because big data collection, and insights resulting from this collection, spans the Internet and Internet-enabled devices through social media listening and posting, sponsored blogs, and other forms of sponsored partnerships. Lucas recognized that the industry is "blurring the lines" in what practices are carried out within advertising agencies: "What used to be advertising, could now in some ways be considered public relations." Again, the challenges surrounding the consolidation of workplace responsibilities that leads to more work for practitioners was a worry among the practitioners interviewed, which is exacerbated by the lack of qualified talent.
Based on the participants' responses, there appears to be a recipe for burnout in data analytics due to industry expectations. Being overworked seemed more prevalent at the smaller, niche agencies, as there is consolidation of resources, including human resources. Thus, this leads to the second part of the second emergent theme where practitioners felt a sense of insecurity because they do not fully realize and appreciate their expertise and skill in sorting through data. During recruitment for this study, all participants were briefed on the goal of the research study—to understand practitioners' perceptions toward and experiences with big data. Even with this being the case, those who were not members of large firms felt they did not actually involve themselves with "big data," and chose instead to talk about the large data sets they are exposed to every day. Craig, an analytic strategist, displayed his uneasiness with the term big data at the beginning of the interview:
It's a term I don't actually use myself, but I don't chastise others when they do. . . . So, full disclosure before we go any further into this—other than my definition of big data I can't say that I actually work day-to-day with big data. That being said, everybody that I work with looks at me as the "big data guy," so I think that's because I have a burden of knowing what it is or how I view it.
He showed his confusions about big data throughout the interview, with phrases like, "Well, maybe that is actually big data I work with . . ." A few other participants were hesitant when asked about their specific uses of big data, claiming they "did not know if this is the right answer" when explaining their responses. No one explicitly expressed their lack of confidence around big data, but it was clear through the tone in their responses and explanations of the term big data that some were more comfortable with the topic than others.
According to the practitioners interviewed, large technology firms determine what is big data and the rest are left feeling like they need to catch up due to the lack of resources and available talent, despite these practitioners having quite a bit of expertise in the area already. This belief was so prevalent that smaller agencies were convinced they were not actually a part of the big data realm, even though they were using many large data sets to discover actionable insights, which they would report to clients. The mindset that the organizations were not doing enough to enter into the "big data realm" was difficult to address due to the evolving industry and the presence of more and more accessible data. Those organizations that were not built on the Internet were just trying to keep up with the ever evolving nature of advertising. Access to data was not the issue; however, figuring out how to succeed in the same market as Amazon and Facebook, and adjusting according to those expectations was proving difficult for those who were not members of the trend-setting companies. More companies are turning to machines with adaptive learning to bridge this systemic gap. While big data applications were the central topic of our conversations with practitioners, a discussion of the relationship between humans and machines, a critical societal issue related to big data, was naturally introduced by the participants.
Theme #3: Humans vs. Machines
As participants had noted, the reliance on big data occurs because of advancements and innovations in various technologies that collect, store, and interpret data that users generate when they are using digital devices, such as their computers and smartphones. There is much debate in society today around the increased reliance on automation versus human power, and more specifically whether automation will actually hurt the industries adopting these technologies (Bessen 2016; Thompson 2015). The Atlantic featured two pieces highlighting opposing views on this topic—one arguing technology actually helps grow adopting industries (Bessen 2016), and the other arguing that technology decreases overall employment (Thompson 2015). This debate extends to the advertising world as well. Participants recognized the need for more machines, as they are effective at managing data much faster than what is capable by humans. On the other hand, machines leave out the "human" element—insight comes from human interpretation of the data. This not only reflects the long-lasting discussion of technological affordances and technological determinism (Schroeder 2014), but also multi-layered dilemmas surrounding big data (Ekbia et al. 2015). Basically, the technological affordances viewpoint emphasizes the possibility that human beings can actively make use of technologies, while the technological determinism perspective focuses on the power of technology shaping society. The developments of big data have highlighted and intensified these two distinct perceptions. In addition, these developments have also brought epistemological, methodological, aesthetic, technological, legal, ethical, and political economy dilemmas (Ekbia et al. 2015). In this mix, though, Jordan's company was actually developing machine learning to mimic human decision-making, allowing for faster and more efficient data processing and interpretation: "The benefit of the machine learning—which is what is really happening in our world—you know, it just does it in seconds versus minutes or hours or days. So the time, it's a speed market. It's getting that information and telling us what to do immediately so we can act on it."
The sentiment of those interviewed was generally that machines are one tool that can be used by practitioners. Ultimately, all decisions were being made by humans. In other words, although technology provides affordances and make many things possible such as constantly tracking consumers' behaviors and accurately profiling target audiences (Schroeder 2014), human beings are still in control and lead strategic decisions in advertising practices. Ben, director of planning, backed this up by saying, "For the work we do, the strongest performing work is always based on real human insight and the ability to connect with people." Machines allow participants to analyze large sets of data quickly, but interpreting the data was where experienced humans were an organization's most valued resource because humans know how to assess the data to find actionable insights. Christine, a managing director of research, insight, and strategy, talked about the importance of business questions, as these helped guide the interpretation process that suggests actionable insights: "Just trying to come up through data and look for something interesting—it's hard because you need to know what is or isn't interesting."
Kathy, an industry consultant, went a step further and noted that the best data scientists should know more than just how to run analyses. This is where machines cannot replace humans. It is integral that data science students are taught how to translate the numbers into useful insights because "that's the most important part!" As she states,
. . . I feel in some ways we've gotten less connected to people who we're supposed to move. . . . That's one of my concerns. When we think about data driving decisions, it shouldn't be about data—it should be about insights, understanding that data has limitations a numbers can't necessarily represent. . . . It is certainly something you can observe, but it doesn't tell you the "why" behind things.
Much of this topic on the debate of humans versus machines revolved around the idea that machines are a great tool, but they are not effective without humans. Lucas, a CEO, highlighted this point by comparing big data to music: "The raw data is the sheet of music. Not all musicians will play that music the same way. Even though all the notes are there and all the markers are there . . . every musician plays a little bit different. And that's what makes a symphony." So, while business and society are moving toward heavier reliance on machines (e.g. using devices such as Google Home to control multiple appliances with one's voice), those in advertising believed the best decisions bring a true human element to the mix, as advertisers are in the business of selling to humans, after all. Humans create the machines and the algorithms for the software that run the machines, and they then actually perform the analyses, interpretations, and insights. Essentially, humans cannot be removed from this process and practitioners recognized the need to capitalize on the strengths of both employee skills and machine capabilities on data collection, analysis, and interpretation.
Theme #4: What Is "Big Data" Anyway?
The goal of this article was initially to explore the concept of big data within the advertising industry from the perspective of those who actually use the data to make strategic decisions. While participants believed there were many different definitions and perspectives when describing big data, there was consensus in that big data is an accumulation of many sets of "small" data. Small data in the context of this study refer to data outside of the realm of big data. The key characteristics that differentiate big data from small data are the three "V's" which are volume, variety, and velocity (Gandomi and Haider 2015). Compared to small data, big data have large sizes, heterogeneric structures in a data set, and are used to quickly generate data run large-scale analyses. Despite the difference in naming these data sets, practitioners did not view them differently. Data were tagged as "big" when many sets were combined to form one analysis, such as when Lucas' organization used US Census data to create consumer profiles for retail stores based on previous purchase behaviors.
When it came down to defining the term big data, a few practitioners began to question whether it should be named as such. Scott believed the emphasis and hype around the term made it a "bad word because small data is vital too." In the industry, this term seems to be meaningless without context or specific application. Christine, who worked at a large tech firm as a managing director of research, insight, and strategy, was not convinced it has any meaning: "I think it's a buzzword that a lot of people use and we don't know what it means . . . I think big data is just a marketing term. It's not a real thing, right? It's just a proliferation of—more data is being collected about things that we weren't collecting before."
Part of this problem of collecting more data than before was mentioned by industry consultant Kathy, as well. She explained big data as "Internet data," which included social media metrics and click data, and then followed up with, "I know we have more information than we used to, but I feel in some ways we've gotten less connected to the people who we're supposed to move." Lucas, a CEO, also mentioned that integration was a key aspect of big data, in that "big data is now getting better and better because one data set is being integrated into another data set." A common occurrence was the mention of sources of data sets, as well as how these sets are best utilized to create knowledge of consumers. This knowledge came at the cost of consumer ignorance, however. Ben acknowledged that Facebook's reaction buttons were a method to gathering more information on users without them realizing the platform was collecting information through those interactions. Lucas explained this method with, "consumers just don't really trust ads," which led brands on the Internet to become more creative with their data collection techniques. Practitioners were hesitant to talk about the issues around privacy, but many of them recognized that the idea of advertising with the help of "big data" would stir up negative connotations with consumers.
Overall, for the practitioners interviewed for this study, big data was just a fancy term placed on large sets of integrated data, which is then marketed to clients as "big data." Randy, a director of analytics, believed a data set was made up of big data when an organization needed specialized tools to analyze the data and turn it into insights. To consumers, these practices manifest into Buzzfeed quizzes and Facebook reaction buttons, but to advertising practitioners, those actions were worth money. Big data should be more than just numbers, though, as "it means nothing unless you know how to interpret it and draw inferences from it and then be able to actually apply those inferences" (Lucas, CEO).
Overall, advertising and marketing practitioners recognize the potential of big data in advancing the industry's knowledge of consumers, but "more" does not necessarily always mean "better" when it comes to decision-making and insight. Learning how to most effectively use the large amounts of available data in today's advancing world is a unique advantage based on an organization's resources and talents. However, the uneven control and distribution of resources and talents may also create a "big data divide," which may lead to unfair competition and cause other related social issues such as data insecurity and data discrimination. Data insecurity refers to the problem of intentional or accidental destruction, modification, or disclosure of data; and data discrimination is the bias that occurs when predefined data types or data sources are intentionally or unintentionally treated differently than others.
This study contributes to our understanding of advertising and society through an indepth examination of advertising and marketing practitioners' perceptions toward and experiences with "big data." With little research exploring practitioners' feelings and thoughts about big data in their work and decision-making, this study's interviews allowed participants' voices to be heard through their direct experiences with big data. Consistent with previous literature (boyd and Crawford 2012; Fulgoni 2013; Goes 2014; Kim 2014; Kitchin 2013, 2014), participants in this study also considered "big data" to be a fluid term that signals different meanings in different contexts. According to their understanding, the actual data being used in the industry does not differ from other types of data, but it is an accumulation of many data sets that turn "small data" into "big data." In other words, there is no distinctive difference between data, small data, and big data—big data is built on, and constructed by, small data. This finding indicates a shift in the way data is viewed in the industry (Kierlanczyk 2016; Winshuttle 2019).
The largest contribution this study makes to the advertising literature is the four themes generated around the current issues faced by practitioners attempting to collect, analyze, and interpret sets of big data, which enriches our understanding of big data in the unique context of advertising from a practitioner's inside perspective. Biesdorf, Court, and Willmott (2013) addressed this concern by recognizing that the key to success with big data analytics is "simply put, developing a plan." While previous studies have collected initial evidence of the applications of big data in marketing fields (Culotta and Cutler 2016; Liu, Singh, and Srinivasan 2016; Trusov, Ma, and Jamal 2016), from this study's interviews, big data is most effective when humans and machines work together to create actionable insights. Machines are a great resource, but the bigger issue stems from the lack of enough skilled people in agencies to work with big data. As mentioned previously, this finding well reflects the long-lasting debate on technological determinism. While new technologies generating, collecting, storing, and processing big data are an important source for changes in society, human beings are still in control and are a key factor to drive the changes in generating relevant and actionable insights.
Some of the practitioners interviewed in this study may have displayed uncertainty around their work with big data, but they knew the importance of the outcomes of the large sets of information they use on a daily basis. From this, advertising agencies should instill more confidence in the minds of strategists and analysts to feel more comfortable with "big data" and their role in creating reports using it. If smaller traditional advertising organizations wish to compete with larger digital firms, they must place emphasis on hiring individuals who have experience with big data or larger data sets (Biesdorf, Court, and Willmott 2013). Gaining a more complete picture of consumers now requires knowledge in this area, or you will be left behind in this data-driven world. Even more important seems to be investing in continuing education for current employees. Those practitioners who were not members of large technology firms felt they were always a step behind groups like Facebook and Amazon, mostly due to their perceived inability to effectively analyze very large data sets. Some were attempting to solve the issue through machine learning, but most recognized there are simply not enough people who possess data science skills within the marketing and advertising profession.
Ultimately, it is important to remember that more data does not automatically equal the best data. If advertising agencies inundate analysts and strategists with irrelevant data, or cannot provide adequate resources for analysis and interpretation, the efficiency of using big data can decrease dramatically. The practitioners interviewed for this study believed small data still had its purpose in understanding consumers and their preferences. Knowing how to best integrate all of appropriate "small" data sets to build insight about a target group is usually more important and can earn more credibility than just simply using "big data" as a buzzword to attract clients.
LIMITATIONS AND FUTURE RESEARCH
Several limitations should be noted. Although the intention of this study was to explore advertising practitioners' perceptions and understandings of big data, only a limited number of advertising practitioners were recruited. Future research may recruit a more diverse group of advertising practitioners to compare with the current study and make more generalizable results for the industry at large. Similarly, while the generalizability is not a concern for qualitative research, it would be nice to know whether or not findings of the current study are supported by advertising practitioners in the industry or other similar contexts to see any commonalities and differences across contexts. Therefore, future research may design a large-scale survey to verify the findings generated from the current study. Furthermore, the current study mainly focused on advertising practitioners' general understandings of big data. As the interviews revealed for this study, the applications and practices of big data are diverse and many questions remain unanswered. Future research could explore specific aspects of big data application and development such as effective usage of small data and big data, and, optimum relationship between machines and human brains to obtain a more complete picture of big data marketing. Finally, with the global penetration of big data, future research may be conducted in different cultural contexts beyond the United States to compare with the current study to see possible similarities and differences across many contexts. Finally, the central focus of the current study is advertising practitioners' daily experiences with big data. While our participants touched on some societal and ethical issues caused by big data, their discussion of ethics lacked breadth and depth. Future research may examine how advertising practitioners perceive big data in relation to ethical, societal, and cultural issues in the industry and society to generate more insights.
Ekbia et al. (2015) indicated that all digital technologies are dual in nature: on the one hand, technologies are empowering, liberating, and transparent; on the other hand, technologies could also be intrusive, constraining, and opaque. Big data, as the embodiment of the latest advances in digital technologies, vividly display this dual character. As our findings suggest, according to the advertising practitioners interviewed in this study, big data greatly enhance the effectiveness and efficiency of consumer communications and facilitate strategic decision-making. At the same time, big data also cause burnout and feelings of insecurity among those practitioners. Furthermore, the developments of big data intensify the discussion of the complicated relationship between human beings and technology. Due to the complexity and ubiquity of big data, a concerted conversation between academics, practitioners, and policy makers is needed to have meaningful conclusions about how big data should be collected and used. Our study adds to the conversation by revealing challenges and issues facing advertising practitioners when dealing with big data in their everyday practices, as well as their perceptions of possible societal and cultural issues caused by the interplay between advertising practices and big data.
Huan Chen is an associate professor of advertising in the College of Journalism and Communications at the University of Florida. Her research interests include new media and advertising, product placement, international and cross-cultural advertising, and social media and health communication. She has published more than 30 refereed journal articles. Her research papers have appeared in Journal of Advertising, International Journal of Advertising, Journal of Business Research, and Health Communication, among others. She has also published the book Connecting Virtual World Perception to Real World Consumption: Chinese White-Collar Professionals' Interpretation of Product Placement in SNSs (VDM Verlag, 2011) and four book chapters. She received her PhD in communication and information from the University of Tennessee at Knoxville.
Brittani Sahm is a visiting assistant professor of communication at Rollins College in Winter Park, Florida, where she focuses her teaching and research on the intersection of new media and strategic sport messaging. Dr. Sahm is particularly interested in the societal implications of sport media messaging, specifically examining the affective outcomes reported by sport fans as a result of increased sport media consumption. She received her PhD in mass communication from the University of Florida at Gainesville.
Appendix. Interview Guide
Introduction: Thank you for agreeing to meet with me today. As we briefly discussed through emails and telephone, I am currently researching on advertising practitioners' perspectives on big data and the application of big data in advertising practices. Specifically, I am interested in your experiences, feelings, ideas, and perceptions of big data and its application in your everyday advertising practices.
1. Tell me about the current status of US advertising industry.
2. When did you hear the term of big data?
3. Based on your own experiences and understandings, what is big data?
➢ Probe questions: What are the differences between small data and big data? What is the definition of big data from the perspective of advertising practitioners?
4. What does big data mean to the US advertising industry?
5. What are different big data applications in the US advertising industry?
6. Tell me about your experiences with big data applications.
➢ Probe questions: What do you do with big data? How do you collect, analyze, and interpret big data? What are the barriers and challenges of using big data in the U.S. advertising industry? What are the relationships between big data and small data? What are the advantages and disadvantages of using big data?
7. What are some skills that are important for college students to learn now if they want to work with big data in this field?
➢ Probe questions: What are the best ways for them to learn these skills?
8. What are some concerns around the use of and reliance on big data in the advertising industry?
➢ Probe questions: How are practitioners attempting to address these concerns?
7. What do you think the future developments of big data applications in the US advertising industry?
➢ Probe questions: What are the possibilities? How do applications of big data influence the practice and development of the whole advertising industry?
Volume—collection and storage of terabtyes (or more) of consumer data.
Velocity—collection is constant and in real-time.
Variety—vast amount of differing information collected on consumers.
Exhaustive—increased likelihood to examine large portion of consumers within a population.
Indexical—bits of data are easily identified and retrieved within system.
Relational—ability to conjoin and compare differing collections of similar data.
Flexible—ability to navigate collection and interpretation of data fluidly.
2. "Iditarod-style" refers to the shape of the conceptual framework established by Jayaram et al. (2015). The name originates from the Iditarod Trail Sled Dog Race that takes place in Alaska each year. The framework is shaped like a traditional "sled dog team layout," where each concept is a "dog" and the technology and market characteristics are presented as two rows, although they are all interconnected.
3. Relationship marketing can be defined as the actions taken by a brand to establish and maintain long-term relationships with customers through communication and delivery of benefits related to continued patronage.
4. According to Moretti and Tuan (2014), social media marketing is defined as "the process that empowers individuals to promote their websites by gaining attention through Social Media sites and by tapping into a larger community that may not be available via traditional communication channels" (124).
5. For example, there may be a positive relationship observed between the stock index and butter production: as the stock index increases, butter production is also seen to increase.
6. Continuing with the stock index and butter example, despite an observed positive correlation, it does not mean that the stock market increases cause increased butter production (and vice versa). There are too many other factors that may explain increases in the stock market and butter production. The only way to adequately determine causation is through a controlled experiment.
7. The interviews were conducted in 2015 and 2019.
8. When participants were asked to describe what they meant when referring to work that did not involve data, they described the more "traditional" elements of advertising such as brand-building through copy writing and creative development (e.g. videography).
9. It could be argued that these factors are not necessarily as important to potential job candidates anymore, especially with the appeal of start-up culture and controversial practices at large technology firms (Coleman 2018; Wakabayashi 2017). These interviews were conducted in 2015, before many of these issues were brought to light. Now, we may see different patterns regarding employment choices, but these were the findings at the time.