Big data brawlers: 4 challengers to Spark

Big (and even not so big) data hasn’t been the same since Apache Spark made inroads with developers and became a staple ingredient in big data clouds.

But Spark is far from perfect. It’s certainly improving, as version 2.0 shows, but if a competitor offers a better handle on what Spark does and more, developers will pay attention.

Here are four projects emerging as possible competition for Spark, with new approaches to handling the conventional in-memory batch processing Spark is famous for and the streaming Spark continues to work on.

Apache Apex

What it is: Originally created by DataTorrent, Apex has since been donated to the Apache Foundation. It performs both stream and batch processing on Hadoop under YARN.

DreamHost Works to Restore Email Services After Prolonged Outage

Los Angeles-based web hosting provider DreamHost is continuing to resolve an issue with its email services that started on Monday when it noticed “extremely high loads on [its] homiemail-sub5 and homiemail-sub3 email clusters.” As of this morning, there was still no estimate into when all services would be fully restored.

On its status page on Tuesday, DreamHost informed customers that it had to perform “additional emergency maintenance on two of [its] mail filer systems on the homiemail-sub5 mail cluster of servers.” At the time, it was anticipated to only impact some customers for a “few hours” but the issue lasted days for many customers.

DreamHost has provided a statement to The WHIR by email on Friday:

Security Researchers Allege Russian Ecommerce Firm Turns Blind Eye to Crime

Russian ecommerce shop provider Deer.io is allowing dark web activity out in the open, according to a report from threat intelligence and security analysis firm Digital Shadows.

What is the difference between this case and any other where a customer uses a web host to carry out criminal activity? Digital Shadows alleges that the majority of Deer.io shops sell stolen products or breached data, and that the company advertises on hacker and cybercrime forums.

SEE ALSO: Snowden Blasts Russia’s Proposed Anti-Terror Laws

StackPath: Can Lance Crosby Win Round Two?

On May 6, 2016, Symantec’s CEO at the time, Michael A. Brown said that the internet threat environment will cost our global economy between $2 trillion to $3 trillion annually. This week after a year in pre-op Lance Crosby just launched his new company, StackPath, an internet security company tailored to meet that threat. In building the new business, he has taken a road that I did not expect but one that has a high probability to be successful.

StackPath as a name fairly well sums up the game plan, as well as the title to its website: Intelligent Web Services for Security, Speed and Scale. To deliver these services StackPath acquired four diverse companies in the hosting and security sectors: MaxCDN, Fireblade, Cloak, and Staminus.

The goal is to integrate these firms to provide a cohesive solution. This single unified platform will provide a web application firewall, server content delivery, VPN, API, and DDos mitigation all in one consistent StackPath-branded service.

Amazon Cloud Unit Helps It Stay Profitable While Investing

Big data problem? Don't forget search

With every cool new technology, people get overly infatuated and start using it for the wrong things. For example: Looking through a bazillion records for a few million marked with a set of criteria is a rather stupid use of MapReduce or your favorite DAG implementation (see: Spark).

For that and similar tasks, don’t forget the original big data technology: search. With great open source tools like Solr, Lucidworks, and Elasticsearch, you have a powerful way to optimize your I/O and personalize your user experience. It’s much better than holding fancy new tools from the wrong end.

A bad job for Spark

Not long ago a client asked me how to use Spark to search through a bunch of data they’d streamed into a NoSQL database. The trouble was that their pattern was a simple string search and a drill-down. It was beyond the capabilities of the database to do efficiently: They would have to pull all the data out of storage and parse through it in memory. Even with a DAG it was a little slow (not to mention expensive) on AWS.

Spark is great when you can put a defined data set in memory. Spark is not so great at sucking up the world, in part because in memory analytics are only as good as your ability to transfer everything to memory and pay for that memory. We still need to think about storage and how to organize it in a way that gets us what we need quickly and cleanly.

VeriSign: 12 Million Domains Registered in Q1 2016

The total number of registered domains across all TLDs reached 326.4 million, growing 12 million, or 3.8 percent from Q4 2015, according to the Q1 2016 Domain Name Industry Brief from VeriSign. Year-over-year registrations increased by 32.4 million, or 11 percent.

The number of internet sites redirecting to popular social media and ecommerce sites rose significantly over the past year.

SEE ALSO: New TLDs Growing as Over 3 Million Domain Names Added in Q3: VeriSign

Report: Rising Reliance On Digital Content And Data Sovereignty Drives Data Center Demand

Unprecedented demand for online content such as movies, videos, apps, social media, and photos have become a staple part of the digital diet. And according to JLL’s latest data center report, the adoption of cloud services to store all this content is expected to double the size of the North American data center industry by 2021.

“With information streaming in from every corner of the world, organizations struggle to understand where and how to best manage and process their data to deliver instant, reliable access to information for consumers and businesses alike,” said Bo Bond, Central Region lead for JLL’s Data Center Solutions Group.

“Cloud services, digital content, and new data sovereignty laws are setting the data center market on fire,” he continued. “Demand is historically strong, so the onus is on the data center operators to build space fast enough, while also accommodating shorter, more flexible lease structures that have become highly popular as data strategies have evolved.”

With a spike in protective data sovereignty laws enacted this year, JLL’s annual North America Data Center report also observes that countries are beginning to regulate where the cloud ‘lives.’ Data sovereignty means that digital data is subject to the laws or legal jurisdiction of the country in which it is stored. With this some of the industry’s biggest players are expanding globally faster than ever to meet growing demand and regulatory compliance requirements.

The JLL report covers 17 North American markets for data center facilities, and reveals the top trends influencing U.S. data center locations, including:

1. Cloud adoption will double the size of the data center industry over the next five years. Cloud adoption is racing ahead at break neck speed and even this year, JLL has seen record absorption of data center space totalling 274.2 MW. This is spurring a development surge across the country, especially in markets like Northern Virginia where 1.13 million square feet is under construction and a further 2.67 million square feet is planned.

2. Data center users are disbursing data across locations, aligning with smarter data management strategies. Data center providers and users alike are getting smarter about location planning and load management, while pursuing shorter, more flexible lease structures.

3. Data sovereignty laws are redrawing the global data center location map. From Brazil to Russia, the industry’s biggest players are expanding internationally faster than ever to meet growing demand and help users stay compliant with regulations designed to keep data inside a nation’s borders.

4. Climate change is shaping data center legislation and technology. The realities of global climate change have spurred effective energy efficiency solutions, from refrigerant-based cooling systems to the continuing rise of data center micro grids.

“Cloud services will continue to be the primary demand driver in North America,” said Bond, “The industry’s biggest players are ready for it. They are upping their game with smarter capacity planning and virtualization strategies to meet the growing demand for this very specialized space.”

Source: CloudStrategyMag

HCSS Chooses WaaS Platform From CloudJumper

CloudJumper has announced that construction industry software leader, HCSS, has standardized its cloud-based construction software solutions on the CloudJumper nWorkSpace WaaS platform. A pioneer in heavy construction software, HCSS leverages CloudJumper to provide the company’s HeavyBid*, HeavyJob*, and other applications in a hosted environment. Additionally, HCSS utilizes CloudJumper’s platform to deliver complete IT workspaces that allow customers to access critical business applications.

HCSS’s applications support thousands of construction users across the United States in streamlining their operations with cloud-based access to HeavyBid and HeavyJob. The company’s software applications help construction companies better manage their projects from bid to completion. Prior to CloudJumper, HCSS built an in-house solution to make HeavyBid and HeavyJob available to its customers via the cloud. While they were running a successful platform, the company’s exponential growth over a short period of time soon created challenges regarding scalability and speed of deployment. To address the issue, HCSS chose the CloudJumper platform with its service-oriented infrastructure to build a full-featured cloud application software offering capable of meeting user demand.

“We have been growing tremendously and CloudJumper has been growing with us which is what we look for in a business partner,” said Ketul Parekh, cloud services manager, HCSS. “This is very strategic technology for HCSS, requiring a strong partnership to ensure our mutual success. CloudJumper is always there as a partner, and that is an important part of their value.”

“During our review of alternate vendors, it was clear from the evaluations that CloudJumper offered the most mature platform,” added Parekh. “The executives and support team at CloudJumper understood what we needed to meet our goals and worked closely with us to develop the right cloud solution for our software. Having standardized on the CloudJumper nWorkSpace platform, the technology has proven itself in cloud-enabling our applications without the need to redevelop existing software.”

HCSS customers benefit from reduced costs by eliminating the need to invest capital in new servers or IT staff to support application servers and associated infrastructure. Applications are deployed in an enterprise-class environment that includes data backup and redundancy for data protection and business continuity which greatly simplifies IT management. The solution also allows HCSS to target smaller customers as the barriers of entry are lower for deploying the application in a hosted environment compared to a traditional on-site or self-hosted model. The automation, ease of deployment, and scalability of nWorkSpace have been crucial to the solution’s success in helping HCSS deploy software quickly. This has been a win for HCSS and its customers.

CloudJumper markets its solutions through a robust white label program and simplifies management for partners with its easy-to-use control panel, a single pane-of-glass provisioning system, allowing for complete control over customer installations. CloudJumper operates a network of autonomous data centers, all connected by a highly available fiber-optic network. From these locations, the company provides cloud computing solutions with inherent geographical diversity for customers.

“We have made the road to the cloud a simple and cost-effective process for ISVs seeking a highly scalable platform to bring their software to customers globally,” said JD Helms, president, CloudJumper. “As illustrated in the case with HCSS, the level of workflow and automation we provide significantly reduces the complexity and cost of cloud-enabling entire catalogs of software in a secure, highly reliable platform.”

Source: CloudStrategyMag

Report: Cloud Data Security Still A Challenge For Many Companies

Despite the continued importance of cloud computing resources to organizations, companies are not adopting appropriate governance and security measures to protect sensitive data in the cloud. These are just a few findings a Ponemon Institute study titled, “The 2016 Global Cloud Data Security Study,” commissioned by Gemalto (Euronext NL0000400653 GTO), the world leader in digital security. The study surveyed more than 3,400 IT and IT security practitioners worldwide to gain a better understanding of key trends in data

According to 73% of respondents, cloud-based services and platforms are considered important to their organization’s operations and 81% said they will be more so over the next two years. In fact, 36% of respondents said their companies’ total IT and data processing needs were met using cloud resources today and that they expected this to increase to 45% over the next two years. 

Although cloud-based resources are becoming more important to companies’ IT operations and business strategies, 54%% of respondents did not agree their companies have a proactive approach to managing security and complying with privacy and data protection regulations in cloud environments. This is despite the fact that 65% of respondents said their organizations are committed to protecting confidential or sensitive information in the cloud. Furthermore, 56% did not agree their organization is careful about sharing sensitive information in the cloud with third parties such as business partners, contractors and vendors. 

“Cloud security continues to be a challenge for companies, especially in dealing with the complexity of privacy and data protection regulations,” said Dr. Larry Ponemon, chairman and founder, Ponemon Institute. “To ensure compliance, it is important for companies to consider deploying such technologies as encryption, tokenization or other cryptographic solutions to secure sensitive data transferred and stored in the cloud.” 

“Organizations have embraced the cloud with its benefits of cost and flexibility but they are still struggling with maintaining control of their data and compliance in virtual environments,” said Jason Hart, vice president and chief technology officer for Data Protection at Gemalto. “It’s quite obvious security measures are not keeping pace because the cloud challenges traditional approaches of protecting data when it was just stored on the network. It is an issue that can only be solved with a data-centric approach in which IT organizations can uniformly protect customer and corporate information across the dozens of cloud-based services their employees and internal departments rely every day.” 

Key Findings

• Cloud security is stormy because of shadow IT.
 According to respondents, nearly half (49%) of cloud services are deployed by departments other than corporate IT, and an average of 47% of corporate data stored in cloud environments is not managed or controlled by the IT department. However, confidence in knowing all cloud computing services in use is increasing. Fifty-four percent of respondents are confident that the IT organization knows all cloud computing applications, platform or infrastructure services in use – a 9% increase from 2014. 
• Conventional security practices do not apply in the cloud.
 In 2014, 60% of respondents felt it was more difficult to protect confidential or sensitive information when using cloud services. This year, 54% said the same. Difficulty in controlling or restricting end-user access increased from 48% in 2014 to 53% of respondents in 2016. The other major challenges that make security difficult include the inability to apply conventional information security in cloud environments (70% of respondents) and the inability to directly inspect cloud providers for security compliance (69% of respondents). 
• More customer information is being stored in the cloud and is considered the data most at risk.
 According to the survey, customer information, emails, consumer data, employee records and payment information are the types of data most often stored in the cloud. Since 2014, the storage of customer information in the cloud has increased the most, from 53% in 2014 to 62% of respondents saying their company was doing this today. Fifty-three percent also considered customer information the data most at risk in the cloud. 
• Security departments left in the dark when it comes to buying cloud services. Only 21% of respondents said members of the security team are involved in the decision-making process about using certain cloud application or platforms. The majority of respondents (64%) also said their organizations do not have a policy that requires use of security safeguards, such as encryption, as a condition to using certain cloud computing applications. 
• Encryption is important but not yet pervasive in the cloud. Seventy-two percent of respondents said the ability to encrypt or tokenize sensitive or confidential data is important, with 86% saying it will become more important over the next two years, up from 79% in 2014. While the importance of encryption is growing, it is not yet widely deployed in the cloud. For example, for SaaS, the most popular type of cloud-based service, only 34% of respondents say their organization encrypts or tokenizes sensitive or confidential data directly within cloud-based applications. 
• Many companies still rely on passwords to secure user access to cloud services.
 Sixty-seven percent of respondents said the management of user identities is more difficult in the cloud than on-premises. However, organizations are not adopting measures that are easy to implement and could increase cloud security. About half (45%) of companies are not using multi-factor authentication to secure employee and third-party access to applications and data in the cloud, which means many companies are still relying on just user names and passwords to validate identities. This puts more data at risk because 58% of respondents say their organizations have third-party users accessing their data and information in the cloud.

Recommendations for Data Security in the Cloud

The new realities of Cloud IT mean that IT organizations need to set comprehensive policies for data governance and compliance, create guidelines for the sourcing of cloud services, and establish rules for what data can and cannot be stored in the cloud. 

IT organizations can accomplish their mission to protect corporate data while also being an enabler of their “Shadow IT” by implementing data security measures such as encryption that allow them to protect data in the cloud in a centralized fashion as their internal organizations source cloud-based services as needed. 

As companies store more data in the cloud and utilize more cloud-based services, IT organizations need to place greater emphasis on stronger user access controls with multi-factor authentication. This is even more important for companies that give third-parties and vendors access to their data in cloud.

Source: CloudStrategyMag