T-SQL Tuesday #95: Big Data

T-SQL Tuesday is a monthly blog party started by Adam Machanic (t|b). This month’s T-SQL Tuesday is being hosted by Derek Hammer (t|b).  Everyone is invited to this party. Participation is easy. Each month, members of the community write a blog post about the topic presented and post it on the second Tuesday of the month.  This month I was getting this together, but I want to participate.  Better late than never.

This month’s topic is Big Data.  Big Data is a booming area of technology. Technical professionals and companies alike are investing a lot in Big Data.  Posts can be about; how big data affects the industry and our careers, how the cloud is enhancing our ability to work with big data, how you deal with big data in SQL Server on-premises, NoSQL, development challenges and strategies for working with internet of things data, or anything else you come up with. Big data has become quite large (pun intended) and should offer a lot of freedom for self-expression in this month’s posts.

Business is always full of buzzwords with some of the current ones being ‘Deep Dive’, ‘Incentivize’, ‘Outside the Box’, ‘Wheelhouse’, ‘Drill Down’, and “Big Data’.  These are not bad, but instead they are a sign of where business is putting their focus.  Big Data is really an old concept if you consider that processing a lot of data has been going on since computers were invented.  This concept of Big Data has become more popular over the past few years as computer hardware and software has improved data processing abilities.  However, Big Data is not like relational data in that it is not always collected in ordered, relational datasets.  With the increase use of Big Data, databases like MongoDB, DocumentDB, Cassandra, and other NoSQL databases have become popular tools.  These NoSQL databases are designed to store large quantities of non-relational data.

As a SQL Database Administrator, it is important to stay on the leading edge of technology and continue to grow your skillset.  Life (and technology) move pretty fast.  If you don’t keep up, you might be left behind.  The Big Data movement has taken off and it is important for DBAs to understand their role in the industry and how their skillset can grow and be used with the new technologies.  Microsoft has expanded their toolset, giving us new ways to work with Big Data.  Microsoft has made the PolyBase feature available in SQL Server 2016, which was previously available only in the Microsoft Analytics Platform System.

Relational data in conjunction with Big Data are key components of the decision making process.  Hadoop and Azure Blog Storage data can be accessed through SQL Server 2016 and the PolyBase feature allows users to join their Big Data with their SQL Server relational databases.  PolyBase enables the use of existing SQL Server tools, such as SSMS and T-SQL, to query Big Data as well as the use of existing SQL Server tools like Reporting Services, Power Query, and Power BI on the combined data.

Learning more about PolyBase is an opportunity for SQL Server DBAs to learn more about Big Data, how it works with SQL Server, increase their skillset, and become more valuable within their company.  More and more businesses rely on Big Data and Big Data analytics in ways never before dreamed of.  This is just the tip of the iceberg.  There is so much more that can be accomplished with Big Data.  From tweaking marketing campaigns, refining operations, and improving business performance, the uses of big data have not even scratched the surface yet.  Data driven businesses and analytics-enabled decisions will become the norm.  Having the right skills and tools available will pay off in the future as the use of Big Data continues to become the norm.  If you haven’t already done so, install the Microsoft PolyBase feature and become familiar with the product.  The more you know, the more opportunities that might become available.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s