New Survey Finds More Than Two-Thirds of Companies Leave Valuable Data Untapped

Data professionals, hampered by constant infrastructure challenges, struggle to keep up with demands from the business

While data-driven decision making is the number-one indicator of success for today's businesses, a new global study conducted by Dimensional Research for Fivetran, the leading automated data integration provider, shows organizations largely unable to leverage the ever-increasing amount of data they have on hand. According to the study, 44 percent of respondents say that key data is not yet usable for decision making and 68 percent say additional business insights can be extracted from existing data if they only had more time. This is largely attributed to inefficiencies faced by data engineers, who are responsible for transforming data throughout an organization into a format that can be easily analyzed.

Puede leer este artículo en español aquí.

Fivetran is unveiling these and many other findings from an online survey of 543 data professionals across five continents during today's Data Engineer Appreciation Day, a live virtual event that brings together engineers and professionals from around the globe to discuss the state of the industry, share valuable insights, and celebrate data engineers.

Inefficiencies are blamed for data engineers' woes. According to the survey:

Nearly all participants (98 percent) have problems building pipelines to access their data.
98 percent of participants also say their pipelines break, with 51 percent stating it happens more frequently than once a month.
Respondents who reported pipelines breaking daily cite source availability and data schema changes as the top reasons.
As a result, business suffers with data engineers spending too much time fixing and maintaining pipelines, rather than focusing on higher-value tasks that generate a positive impact on the business. 66 percent report reduced operational efficiency and 59 percent report delayed decisions or lost opportunities due to broken pipelines.

"Data engineers are the forces behind revenue-impacting business decisions, yet they are still burdened by the limitations of brittle, hard-to-maintain data pipelines. Companies should be aware of the challenges these valuable employees face so they can deliver reliable data more easily to business stakeholders," said George Fraser, CEO of Fivetran. "Fivetran provides reliable access to data through fully managed data pipelines and SQL transformations — with 99.9 percent guaranteed uptime. This allows companies to put data to work and make decisions faster, instead of spending valuable resources on building and maintaining pipelines themselves."

For large enterprises and small-to-mid size organizations alike, having immediate and reliable access to data is critical for decision-making. The study indicates that pipeline challenges for data engineers continue to strain the process, and businesses lose out on valuable insights as a result.

Additional findings from the survey include:

79 percent of companies plan to hire more data engineers within the next year
72 percent of organizations move source data daily or more frequently
59 percent of companies use 11 or more data sources
91 percent say source availability problems are the reason why pipelines break daily
55 percent say it takes longer than one business day to repair pipelines
86 percent use multiple solutions to build new data pipelines
41 percent say a lack of data strategy (policies, processes, standardized technology) is the top data-related challenge their organization faces

For additional survey insights and images, please visit the Fivetran blog.

About Fivetran

Fivetran, the leader in automated data integration, delivers ready-to-use connectors that automatically adapt as schemas and APIs change, ensuring consistent, reliable access to data. Fivetran improves the accuracy of data-driven decisions by continuously synchronizing data from source applications to any destination, allowing analysts to work with the freshest possible data. To accelerate analytics, Fivetran automates in-warehouse transformations and programmatically manages ready-to-query schemas. Fivetran is headquartered in Oakland, California, with offices around the globe.

Fuente: fivetran.com