Take a closer look at your Spark implementation

Apache Spark, the extremely popular data analytics execution engine, was initially released in 2012. It wasn’t until 2015 that Spark really saw an uptick in support, but by November 2015, Spark saw 50 percent more activity than the core Apache Hadoop project itself, with more than 750 contributors from hundreds of companies participating in its development in one form or another.

Spark is a hot new commodity for a reason. Its performance, general-purpose applicability, and programming flexibility combine to make it a versatile execution engine. Yet that variety also leads to varying levels of support for the product and different ways solutions are delivered.

While evaluating analytic software products that support Spark, customers should look closely under the hood and examine four key facets of how the support for Spark is implemented:

• How Spark is utilized inside the platform
• What you get in a packaged product that includes Spark
• How Spark is exposed to you and your team
• How you perform analytics with the different Spark libraries

Spark can be used as a developer tool via its APIs, or it can be used by BI tools via its SQL interface. Or Spark can be embedded in an application, providing access to business users without requiring programming skills and without limiting Spark’s utility through a SQL interface. I examine each of these options below and explain why all Spark support is not the same.

European Businesses Ill-Prepared for EU General Data Protection Regulation

European businesses are inadequately prepared for looming EU General Data Protection Regulation (GDPR), according to data virtualization and masking company Delphix. The company has released survey results which show a widespread lack of understanding about the GDPR regime set to come into effect in June 2018.

Well over half of UK businesses have little to no familiarity with the tools recommended for GDPR compliance. Even if Britain is no longer part of the EU when the new regulations come into effect, UK companies, like those from around the world, will still need to be compliant to handle EU citizens’ data.

RELATED: European Union’s First Cybersecurity Law Gets Green Light

Aligned Data Centers And Plexxi Partner

Aligned Data Centers and New Hampshire-based Plexxi have announced that they have partnered to deliver an integrated network and data center alternative that can be consumed as a service. The partnership provides the ability to provision data center capacity and connectivity as an on-demand utility.

Data centers and networks today are large upfront investments that require customers to predict future IT needs and either become barriers for growth or are underutilized. In fact, Gartner reports that as much as 50% of data center capacity goes unused. Aligned Data Centers offers cloud providers and enterprises with unique “plug-and-play” pods that are dynamically provisioned to deliver on-demand agility per application. These pods can scale IT resource requirements from a single rack environment to thousands with ease without having to predefine rack space and network capacity requirements. The platform adjusts to dynamic rack loads, significantly increasing the speed by which cloud players can expand and adjust to growth. Aligned’s platform can accommodate up to 10 times the power density of traditional data centers.

“The compliment between our technology platforms is significant and disruptive,” said Aligned Energy CEO Jakob Carnemark. “Cloud players need data center infrastructure that is as agile and frictionless as provisioning servers, storage and software. Our partnership solves for this by allowing customers to provision infrastructure capacity to align with application needs, significantly reducing stranded capacity and improving speed.”

“Aligned Data Centers re-invented traditional data center architecture,” said Plexxi CEO Rich Napolitano. “They’ve built the world’s first true application-defined data center from top to bottom, including power, space and cooling that adjusts based on customer application requirements and densities. This has never been done before, and truly revolutionizes the cost model of building reliable data centers. IT executives are looking to drastically reduce energy consumption in their data centers, and this partnership enhances Aligned’s transformative pay-for-use model.”

In November 2015, Aligned opened its ultra-efficient, next generation data center in Plano, Texas. Construction is well underway on their second in Phoenix, Arizona, which will be one of the largest multi-tenant data centers in Arizona, capable of offering both high-availability and high-efficiency while guaranteeing an industry-leading 1.15 PUE and reducing water consumption by as much as 85% through its patented technology.

Source: CloudStrategyMag

Romonet’s Files Patents For New Analytics Platform

Romonet has announced it is filing for a number of new patents for the next phase of its data center intelligence platform, utilizing the applications of Machine Learning. The company is known as the leader in data center analytics and this development enhances the value of Romonet’s already patented solution.

“We have been working on advanced data handling and Machine Learning algorithms for over a year, focusing predominantly on enhancing our solution to learn and become as proficient as our human data scientists are today at identifying anomalies, and tracking down the cause behind the symptom. This capability provides powerful operational and business insight into data center systems and component level performance,” said Liam Newcombe, Romonet’s co-founder and CTO.

Having modeled, collected data and analyzed hundreds of data centers in the past eight years, Romonet’s platform has an incredibly detailed and expansive data archive on how facilities of every size perform under different climate, environmental, energy, IT and commercial factors.

While the use of Machine Learning applications is not new, Romonet’s platform is the first to combine metered data, Machine Learning, simulation and predictive analytics.         

“In our case, teaching the machine is much faster as we feed it pre-cleansed and calibrated data to recognize and learn patterns, incorporate additional data from outside sources, and teach the software to suggest causes and recommended actions from previously learned results,” continued Newcombe.

Like Google, Amazon, Cisco, and Netflix, who already use Machine Learning to personalize services and business intelligence, Romonet’s industry-leading platform is revolutionizing the booming global data center market.

With Romonet, Hyperscale and Multi-Tenant Data Center (MTDC) operators are improving the services they provide to their customers while strengthening financial management through investment, cost and margin analysis. Enterprise data center owners, whose facilities, while core to service delivery, are also a drain on profitability, are rationalizing their investments, accurately planning a hybrid (owned, colocated, and cloud) strategy years into the future, and improving their ability to make socially responsible decisions that impact the environment, shareholders and employees.

Source: CloudStrategyMag

Advantech Releases IoT Device Management Software Platform

Advantech has launched its new WISE-PaaS/RMM version 3.1, an IoT device management software platform. WISE-PaaS/RMM 3.1 is an open standardized IoT software platform for users by applying MQTT, a standard and popular IoT M2M protocol for device and server communication.

WISE-PaaS/RMM 3.1 comes with more than 100 RESTful APIs including, account management, device management, device control, event management, system management, and database management. RESTful APIs create new web services and help integrate functions and data with their management tools. Furthermore, WISE-PaaS/RMM 3.1 will release WISE-Agent source code as open source. WISE-Agent software works on the device side, helping customers to develop their own applications. WISE-PaaS/RMM highly enhances connectivity for hardware, software, devices and sensors, and helps customers to transform their business to include IoT cloud services.

Centralized Management for Big Data Visualization

Based on WISE-PaaS/RMM 3.0 for remote device monitoring and management, version 3.1 offers centralized management and a dashboard builder for data visualization. Customers can develop dashboards to monitor and manage all their connected devices. WISE-PaaS/RMM 3.1 is also integrated with Node-Red which is a “drag and drop” logic editor tool for users to access data and features in WISE-PaaS/RMM 3.1 for device flow and action control management.

High Availability for Large Scalability

To provide a stable and reliable centralized management platform, WISE-PaaS/RMM 3.1 includes server redundancy whereby devices can have direct connection with the security of a back-up server if the main server loses connection. This is designed to make sure data and services auto-sync between the main server and the back-up server. WISE-PaaS/RMM 3.1 also provides a hierarchical server, which supports the main server and sub-server structure at the same time. Users can use the sub-server for local device management and use the main server to collect data from local servers in order to disperse the load on the main server.

Sensor/Device Connectivity for Data Acquisition

To connect devices and consolidate data acquisition, WISE-PaaS/RMM 3.1 provides a handler structure in WISE-Agent for users to extend data acquisition capability to connected devices. Customers can develop routines based on sample code Advantech provides to create handlers for their own devices and application scenarios. Advantech provides two data bases (MongoDB and PostgreDB), SQL and no-SQL, for data storage and easy management.

WISE-PaaS/RMM 3.1, Open Standardization in IoT Software Platform

With WISE-PaaS/RMM 3.1, customers can collect data from devices, equipment, sensors and software, shorten development time and reduce development resources. It helps customers develop new IoT application models and it also integrates with Advantech’s IoT Gateway Starter Kit, a ready-to-run package with pre-configured hardware, platform services (WISE-PaaS) and Microsoft’s Azure service certification.

Source: CloudStrategyMag

Telstra Acquires Readify, Adding Enterprise Cloud App Expertise

Australian telecom Telstra has acquired software developer and Microsoft services provider Readify, according to an announcement on Monday. Terms of the deal were not disclosed.

Readify builds apps for customers, but also has a Microsoft focus, and has helped build some of the largest Azure implementations in Australia. Readify architects, implements, and supports products like SharePoint and Office 365, and has received several awards from Microsoft.

SEE ALSO: Vocus to Buy Australian Fiber Network, Projects for $637 Million

Alibaba's Cloud Unit Brings on Microsoft Veteran to Lead Big Data Team

Alibaba’s cloud computing unit has hired Microsoft veteran Jingren Zhou to lead its big data and artificial intelligence research teams, who develop platforms to help customers analyze big data, according to a report by the Wall Street Journal on Wednesday.

While at Microsoft, Zhou worked on the Bing search infrastructure team and big data unit as its chief scientist. At AliCloud, he will lead teams in Hangzhou, Beijing, and Seattle, according to the report.

SEE ALSO: Baidu Creates Own Indexes to Paint Picture of China’s Economy

European Union's First Cybersecurity Law Gets Green Light

By Jonathan Stearns

(Bloomberg) — The European Union approved its first rules on cybersecurity, forcing businesses to strengthen defenses and companies such as Google Inc. and Amazon.com Inc. to report attacks.

The European Parliament endorsed legislation that will impose security and reporting obligations on service operators in industries such as banking, energy, transport and health and on digital operators like search engines and online marketplaces. The law, voted through on Wednesday in Strasbourg, France, also requires EU national governments to cooperate among themselves in the field of network security.

Microservices: What They Mean and How They Impact the Channel

“Microservices” is fast becoming one of the newest buzzwords that IT decision makers need to know as DevOps redefines modern software application delivery. Here’s a primer on what microservices mean and how the concept is affecting the channel.

If you’re a programmer, you’re probably already familiar with the idea of microservices. As the term implies, the concept centers around breaking down software into small parts that interact with one another in modular ways.

If that sounds like the design principles behind Unix-like operating systems, or object-oriented programming, it’s because it is. Microservices are really just an expansion of these programming practices into a broader context.

IDG Contributor Network: What matters more: Controlling the Internet's wiring, or its data? Both

In an interesting move, Facebook and Microsoft have forged an alliance to lay a new fiber optic cable under the Atlantic Ocean. Putting aside the environmental concerns this raises for many (including those of us inhabiting islands), it also raises questions about how control of the Internet — and the data belonging to its users — is basically a prize in a multi-strand tug of war involving technology companies and broadband service providers from here to Spain and back.

So, why does it matter who controls the building of Internet infrastructure? How does that relate to who controls user data? And what’s the implication for businesses that rely on Internet technology to deliver their products and services?

Google — part of Alphabet, Inc. — organizes the world’s data and, it could be argued, knows more about the average person than the average person knows about their closest friends and relatives. The company also accounts for more than 10 percent of all advertising spend globally. In holding the most information on, in, and about the Internet, Google could be the most powerful company on the planet.

Google also is involved, through one of the Alphabet companies, in the delivery of Internet services. The company has been experimenting with and investing in satellite and balloon technology (Project Loon) that could deliver Internet access to even the remotest regions of Earth — just as its existing Google Earth satellite project delivers images of those same regions, as well as eerily accurate photography of addresses such as your own.