GitHub - MicrosoftResearch/Dryad: This is a research prototype of the Dryad and DryadLINQ data-parallel processing frameworks running on Hadoop YARN.

This repository has been archived by the owner on Jun 17, 2024. It is now read-only.

MicrosoftResearch / Dryad Public archive

Notifications You must be signed in to change notification settings
Fork 55
Star 325

This is a research prototype of the Dryad and DryadLINQ data-parallel processing frameworks running on Hadoop YARN.

microsoftresearch.github.io/dryad/

Apache-2.0 license

325 stars 55 forks Branches Tags Activity

Star

Notifications

Branches Tags

Name		Name	Last commit message	Last commit date
Latest commit History 28 Commits
ClusterInterface		ClusterInterface
DryadLinqGraphManager		DryadLinqGraphManager
DryadLinqTests		DryadLinqTests
DryadVertex/VertexHost		DryadVertex/VertexHost
GraphManager		GraphManager
JobBrowser		JobBrowser
LinqToDryad		LinqToDryad
LocalScheduler		LocalScheduler
ProcessService		ProcessService
XmlDoc		XmlDoc
samples		samples
.gitattributes		.gitattributes
.gitignore		.gitignore
Dryad.sln		Dryad.sln
Install.ps1		Install.ps1
LICENSE.txt		LICENSE.txt
Microsoft.Research.Dryad.nuspec		Microsoft.Research.Dryad.nuspec
Microsoft.Research.Dryad.targets		Microsoft.Research.Dryad.targets
README.txt		README.txt
SharedAssemblyInfo.cs		SharedAssemblyInfo.cs

Repository files navigation

Dryad

This is a research prototype of the Dryad and DryadLINQ data-parallel 
processing frameworks running on Hadoop YARN.  Dryad utilizes cluster 
services provided as part of Hadoop YARN to reliably execute 
distributed computations on a cluster of computers. DryadLINQ provides
the LINQ programming model for distributed data processing and leverages 
Dryad for reliable execution.

If you just want to run a sample DryadLINQ program, the simplest way to get started
is to follow the instructions at 
http://microsoftresearch.github.io/Dryad/?topic=html/e992fd94-c956-481d-82e6-dbdf45daa722.htm

Requirements

Visual Studio 2013
An Azure HDInsight 3.1 cluster or a Windows YARN cluster composed of x64 machines

Building Dryad

1) Clone the Dryad git repository.
2) Use Visual Studio to open The Dryad solution file (Dryad.sln) located 
   in the root of the repository and build the solution.  The dependencies
   will be fetched from NuGet.