Grid, Cluster si Cloud Computing

Cursul doreste initierea studentilor in teoria si apliactiile de baza din domeniile de grid, cluster si mai ales cloud computing. Se vor studia atat bazele teoretice cat si tehnologiile emergente pe temele legate de sisteme de operare grid si cloud, sisteme de comunicatii si arhitectura de servicii cloud.

Virtualizare si Platforme de Virtualizare slide-uri, slide-uri 2014

09 mai 2017- 16 :00-19 :30 (C335)- Invitat Tudor Damian

Prezenta obligatorie (pt nota la examen)

02/03/2015 18:30-Lambda Architecture/Apache/Kafka/GraphX
in loc de curs inscrieti-va aici si participate.
Inscriere MeetUp

28/03/2018 Platforma Azure (automatizare)

Campus C310 16:00-19 :40 Invitat Florin Loghiade

Prezentare Cloud Computing - Platforma Microsoft Azure si aplicatii Cloud

Campus C310 - 18 aprilie 2014 si 9 mai - ora 16:00 - invitat Radu Vunvulea

Azure overview

Windows Azure Compute

Windows Azure Storage

Prezenta obligatorie (pt nota la examen)


Continut

1.      Introducere in cluster computing: definitii, roluri, taxonomie

2.      Procesare distribuita

3.      Arhitectura, hardware, protocoale pentru tehnlogii cluster

4.      Sisteme de fisiere ierarhice/distribuite

5.      Tehnologii de virtualizare

6.      Procesare grid si cloud - introducere, problematica

7.      Arhitectura sisteme grid si cloud

8.      Implementarea mecanismelor de planificare si partitionare aplicatii

9.      Paradigma Map Reduce

10.  Servicii web

11.  Microsoft Azure

12.  Sisteme de gestiune a datelor in cloud

13.  Amazon Web Services

14.  Google App Engine

Bibliografie:

1.      Foster, Ian; Carl Kesselman (1999). The Grid: Blueprint for a New Computing Infrastructure. Morgan Kaufmann Publishers. ISBN 1-55860-475-8

2.      Li, Maozhen; Mark A. Baker (2005). The Grid: Core Technologies. Wiley. ISBN 0-470-09417-6

3.      G. Reese, Cloud Application Architectures: Building Applications and Infrastructure in the Cloud, O'Reilly, 2009, ISBN:978-0-596-15636-7

4.      Tanenbaum A.S. Operating Systems Design and Implementation (Third Edition). Prentice 2006.

5.      J.F Kurose, K. Ross, Computer Networking - A top down approach, Addisson Wesley, 2007 (4th ed)

6.      Anil Desay, The Definitive Guide to Virtual Platform Management, 2010, Ca technologies, download http://nexus.realtimepublishers.com/dgvpm.php

7.      R. Jennings, Cloud Computing with the Windows Azure Platform (Wrox Programmer to Programmer), Wrox, 2009, ISBN: 978-0470506387

8.      D. Sanderson, Programming Google App Engine Build and Run Scalable Web Apps on Google's Infrastructure, O'Reilly, 2009., ISBN:978-0-596-52272-8

9.      Andy Oram (ed), Peer-to-peer Harnessing the power of disruptive technologies, O'Reilly, 2001, ISBN: 978-0596001100

10.  * * *, http://code.google.com/intl/ro-RO/appengine/docs/

 

1.    Laboratoare/Seminarii

 

1.     Sem 1, 2,3 - Infrastructura Hadoop -instalare si configurare/lansare aplicatii

      Instalare si configurare masini virtuale

      Instalare hadoop ->VM

      Configurare nod master si noduri compute/DFS

      Replicare configuratii si constructie cluster 

Un eexmplu de masina virtuala se poate copia de aici (Hadoop local).(Windows explorer \\172.30.33.1\GCCC-CloudComputing)=> hadoop-centos.7z

Dupa acest laboratoare/seminarii trebuie sa aveti configurat (optional) eclipse cu plugin-ul de hadoop, impreuna cu masina virtuala si sa va puteti conecta din eclipse la masina virtuala pentru a face deploy de job-uri si sa copiati fisiere de date in sistemul distribuit de fisiere hdfs.

2.     Sem 4 - Implementare aplicatie simpla care sa foloseasca cluster-ul WordCount (curs 5)

3. Sem 5,6 implementare aplicatie Azure (Compute/Storage/ folosing Web Roles si Worker Roles) GuestBook (curs 10/11)

4. Sem 7 Prezentare proiect semestru - Hadoop - aplicatie index inversat.

Folosind masina virtuala Hadoop de la adresa de mai sus impreuna cu mediul eclipse si plugin-ul de eclipse sa se scrie un index inversat pentru un set de carti (se pot descarca de pe project gutenberg -alegeti doar formatul text).

Pentru construirea indexului inversat se va tine cont si de o lista de stopwords (cuvinte care nu se indexeaza) care va putea fi citita dintr-un fisier cu acelasi nume (stopwords.txt) sau se vor putea da ca argument in linia de comanda. Un index inversat contine pentru fiecare cuvant distinct lista fisierelor in care aceasta apare ( fisier#1, linie#1, linie#2,) (fisier#4, linie#1, linie#2, ) ) . La rularea aplicatiei trebuie sa puteti sa constituiti un cloud format din mai multe noduri: cel putin calculatorul personal impreuna cu serverul pus la dispozitie la curs.

 

 

Adresa IP server virtual master cloud (campus): 172.30.106.33 (user root, pass:root).

Proiecte de Semestru

Fiecare student va implementa proiectul de semestru atat cu Hadoop(Map Reduce) cat si in Microsoft Azure (Map-Reduce/Compute Services). Problema 2015 este : Index inversat.

Bibliografie pt teme accesati acest director

 

Subiecte referat

Examinare

Examen scris (sau  Prezentare referat) 50% + Proiect Hadoop 25% + Proiect Azure 25% + GuestBook 10%= NOTA FINALA

Sunt necesare :

       

      Minim 5 la Prezentare referat/ Examen Scris (in functie de varianta aleasa)

      Minim 5 la Proiecte

 

Proiecte Prezentare

Proiectele se depunin directorul partajat : \\172.30.106.1\Prezentari-Referate-GCCC intr-o arhiva (zip,rar,etc) cu numele de login al studentului ! (Arhiva nu se mai poate sterge sau modifica odata depusa)

 

IMPORTANT:

La prezentarile de referate aduceti :

Se va crea o arhiva cu numele de login de pe SCS cu aceste 3 fisiere care va depune dupa prezentare in directorul windows partajat la adresa 

\\172.30.106.1\Prezentari-Referate-GCCC (copiati link-ul si faceti paste in windows explorer)

Atentie : odata depus fisierul nu mai puteti face nici o modificare la el !

 

Rezultatele finale - aici

 

 

 

NrCrt

Subiect/Tema

Nume

Grupa

Data Sustinerii

Login

Nota sustinere+ref

Proiecte

Lab

GuestBook

Nota

1

MapReduce + demo

Sabin Mihaila

244

04/06/2013

16:00

mssd0653

10

7;1

10

 

2

Hadoop Distributed FileSystem +small demo

Mihut Bogdan

244

04/06/2014

mbsd0654

10

8.5;10

10

 

3

Hadoop I/O +demo

Zaharia Andrei

243

05/06/2013

16:00

zabd0626

 

 

 

 

4

Developping a MapReduce Application +demo

Zota Patricia

244

04/06/2013

16:00

zpsd0666

10

8.5;1

 

 

5

How MapReduce Works +demo

Groza Bianca

244

05/06/2013

16:00

gbsd0642

10

8.5;10

10

 

6

MapReduce Types and Formats + demo

Bostan Cristian

244

04/06/2013

16:00

bcsd0631

10

10;1

10

 

7

MapReduce Features +demo

Gavrilut Voica

(+++eyes)

244

05/06/2013

16:00

gvsd0640

10

10;9

 

10

8

Setting up a hadoop cluster +demo

Apintei Ioana

244

04/06/2013

16:00

aisd0628

10

8;1

10

 

9

Administering Hadoop + demo

Mihaela Palermo

243

05/06/2013

16:00

pmbd0612

9

3;1

10

 

10

Pig+small demos

Halita Diana

244

28/05/2013

hdsd0643

10

9;8

10

 

11

HBase+demo

Lupulescu Alexandru

244

12/06/2013

16:00

Lasd0648

7

7;1

10

 

12

ZooKeeper

Alina Calin

244

05/06/2013

16:00

casd0634

10

9.5;9.5

10

 

13

Hadoop Case Studies

Puscas Andreea

243

04/06/2013

16:00

pabd0614

9

1;1

1

 

 

Biblio- Hadoop The Definitive Guide

 

 

 

 

 

 

 

 

14

Windows Azure Basic Concepts + small demo

Stetcu Gabriel

243

29/05/2013

16:00

sgbd0618

9.5

1;1

10

 

15

WebRole

Maior Miruna-Adriana

244

28/05/2013

16:00

mmsd0651

10

7;1

10

 

16

The Windows Azure Service Model (ch3 +ch4+ch5)

Bunta Adrian

244

29/05/2013

16:00

basd0633

8.25

10;1

10

 

17

Constructing a site with webroles (ch6+7 )

Dudas Daniel

243

29/05/2013

16:00

ddbd0598

8

 

8.5;1

10

 

18

working with blob storage(ch8+9)

Halmaghi Catalin

244

11/06/2013

16:00

hcsd0644

10

8.5;9

10

 

19

Blob case studies (ch10)

Maties Ioana

(de fapt Sub 13)

244

29/05/2013

16:00

misd0652

10

1;1

10

 

20

Working with structured Data (ch 11+12)

Barbu Ruxandra-Maria

244

11/06/2013

16:00

brsd0630

10

 

9;1

10

 

21

SQL Azure (ch13+14)

Ghele Diana

243

29/05/2013

16:00

gdbd0599

9.5

7;1

10

 

22

Processing with worker roles (ch15)

Mihaela Creta

244

11/06/2013

16:00

cmsd0637

10

1;9.5

10

 

23

Messaging with the queue (ch16)

Mincic Ioana

244

29/05/2013

misd0657

10

10;10

10

10

24

Connecting in the cloud with the AppFabric (ch 17)

Ildiko Darko

 

243

04/06/2013

16:00

dpbd0595

6.25

1;1

1

 

25

Diagnostics and tracing in the cloud (ch18)

Rarinca Madalina Sara

243

11/06/2013

16:00

rmbd0616

9

1;1

1

 

 

Biblio - Azure in Action

 

 

 

 

 

 

 

 

26

HiveQL Data Definition and Data Manipulation

(ch4 +ch5)

Chis Sinziana

243

28/05/2013

16:00

csbd0591

7

5.5;1

10

 

27

HiveQL Queries and Views (ch6+ch7)

Pop Ovidiu Dan

244

05/06/2013

16:00

posd0660

9

9.5;1

10

 

28

Hive Schema Design and Indexing (ch8+ch9)

Trif Maria-Mihaela

243

28/05/2013

16:00

tmbd0625

7.5

5;1

10

No ref

29

Hive Case Studies (Ch23)

Marcu Raluca

253

29/05/2013

16:00

mrbd0730

 

 

 

 

30

Hive Installation and Data Types/Formats (ch2+ch3)

Alexandra Ionasc

243

28/05/2013

17:50

iabd0604

8

1;1

10

 

 

Biblio: Programming Hive

 

 

 

 

 

 

 

 

31

Hbase Fundamentals and Getting Started (ch1+ ch2)

Raluca Beian

 

243

28/05/2013

16:00

brbd0588

 

8.5;1

1;1

1

 

32

Distributed HBase, HDFS and MapReduce (ch3)

Todoran Andreea-Teodora

243

04/06/2013

16:00

tabd0623

10

7;1

10

 

33

HBase Table design (ch4)

Vilt Iuliu-Bogdan

244

05/06/2013

16:00

visd0665

9

6.5;1

10

 

34

Extending HBase with coprocessors (ch5)

Miclos Aranka

244

05/06/2013

masd0656

9

7;1

1

 

35

Alternative HBase clients (ch6)

Raul Rene Lepsa

???

04/06/2013

16:00

lrsd0647

10

7;7

8

 

36

Deploying HBase (ch9)

Keresztes Anna

243

11/06/2013

16:00

kabd0605

8

5.5;6

10

 

37

Operating HBase (ch10)

Bota Daniel

244

12/06/2013

16:00

bdsd0632

10

7;6

10

 

 

Biblio: HBase in Action

 

 

 

 

 

 

 

 

38

Sqoop

Sfat Andrei-Erno

244

29/05/2013

16:00

sasd0662

9

7;7

10

 

 

Constructing and testing privacy-aware services in a cloud computing environment: challenges and opportunities

 

 

 

 

 

 

 

 

 

Security and privacy implications of cloud computing: lost in the cloud

 

 

 

 

 

 

 

 

 

Dynamically scaling applications in the cloud

 

 

 

 

 

 

 

 

 

The KOALA cloud management service: a modern approach for cloud infrastructure management

Alexandra Retegan

243

28/05/2013

16:00

rabd0617

8.5

1;1

1

 

 

A perspective on scientific cloud computing

 

 

 

 

 

 

 

 

 

User Provided Cloud Computing

 

 

 

 

 

 

 

 

 

Managing security of virtual machine images in a cloud environment

Danciu Reghina-Claudia

243

11/06/2013

16:00

drbd0594

10

1;7

10

 

 

A virtual cloud computing provider for mobile devices

Magdas Iulia

244

11/06/2013

16:00

misd0649

9.5

1;1

9

 

 

Security audits of multi-tier virtual infrastructures in public infrastructure clouds

 

 

 

 

 

 

 

 

 

Towards a data-centric view of cloud security

 

 

 

 

 

 

 

 

 

Runtime measurements in the cloud: observing, analyzing, and reducing variance

 

 

 

 

 

 

 

 

 

An overview of the Open Science Data Cloud

Diosan Adrian

244

28/05/2013

16:00

dasd0638

9.75

5.5;1

10

 

 

Big data and cloud computing: current state and future opportunities

Ioana Bacila

244

04/06/2013

16:00

bisd0629

7.5;1

8.5;1

10

 

 

Research on cloud storage architecture and key technologies

Chiorean Bogdan

244

 

cbsd0636

7

4;1

10

 

 

An operating system for multicore and clouds: mechanisms and implementation

 

 

 

 

 

 

 

 

 

nt service recommendation system for cloud computing market

 

 

 

 

 

 

 

 

 

Service specification in cloud environments based on extensions to open standards

 

 

 

 

 

 

 

 

 

Using simulations to assess the stability and capacity of cloud computing systems

Magori Tamas Levente

243

11/06/2013

16:00

mtbd0607

10

10;6

10

 

 

How replicated data management in the cloud can benefit from a data grid protocol: the Re:GRIDiT Approach

 

 

 

 

 

 

 

 

 

Cloud-computing-based framework for multi-camera topology inference in smart city sensing system

 

 

 

 

 

 

 

 

 

Applicability of the willow architecture for cloud management

 

 

 

 

 

 

 

 

 

Adaptive resource provisioning for read intensive multi-tier applications in the cloud

 

 

 

 

 

 

 

 

 

Elastic management of cluster-based services in the cloud

Holom Dorina Ionela

243

12/06/2013

16:00

hdbd0600

9

1;1

10

 

 

Multi-domain grid/cloud computing through a hierarchical component-based middleware

 

 

 

 

 

 

 

 

 

Ontological approach toward cybersecurity in cloud computing

 

 

 

 

 

 

 

 

 

Securing elastic applications on mobile devices for cloud computing

 

 

 

 

 

 

 

 

 

D-Cloud: Design of a Software Testing Environment for Reliable Distributed Systems Using Cloud Computing Technology

 

 

 

 

 

 

 

 

 

Cloud service and service selection algorithm research

 

 

 

 

 

 

 

 

 

Software architecture definition for on-demand cloud provisioning

 

 

 

 

 

 

 

 

 

Contract-based cloud architecture

 

 

 

 

 

 

 

 

 

An efficient multi-dimensional index for cloud data management

Oltean Ioana Maria

244

12/06/2013

16:00

oisd0659

9.5

10;6

10

 

 

SLA-Driven Dynamic Resource Management for Multi-tier Web Applications in a Cloud

 

 

 

 

 

 

 

 

 

Performance model driven QoS guarantees and optimization in clouds

Demian Claudiu

243

29/05/2013

16:00

dcbd0596

9

3;9

10

 

 

Achieving high efficient agreement with malicious faulty nodes on a cloud computing environment

 

 

 

 

 

 

 

 

 

A study of transcoding on cloud environments for video content delivery

 

 

 

 

 

 

 

 

 

Resource-level QoS metric for CPU-based guarantees in cloud providers

 

 

 

 

 

 

 

 

 

Energy Efficient Allocation of Virtual Machines in Cloud Data Centers

 

 

 

 

 

 

 

 

 

Social Cloud: Cloud Computing in Social Networks

Preduca Georgiana

243

11/06/2013

16:00

pebd0613

9

1;7

1

 

 

A first look at problems in the cloud

Urda Marius

244

28

Umsd0663

7.5

7;7

10

 

 

Benchmarking cloud-based data management systems

 

 

 

 

 

 

 

 

 

Automated control in cloud computing: challenges and opportunities

 

 

 

 

 

 

 

 

 

Dynamic Load-Balanced Multicast for Data-Intensive Applications on Clouds

 

 

 

 

 

 

 

 

 

Virtual machine contracts for datacenter and cloud computing environments

 

 

 

 

 

 

 

 

 

Authentication in the clouds: a framework and its application to mobile users

Marius Podea

254

29/05/2013

16:00

pmsd0353

9

7;7

10

9

 

Cloud computing paradigms for pleasingly parallel biomedical applications

 

 

 

 

 

 

 

 

 

Storage Management in Virtualized Cloud Environment

 

 

 

 

 

 

 

 

 

On Technical Security Issues in Cloud Computing

Lupas Roxana

243

12/06/2013

16:00

lrbd0606

8

1;1

1

 

 

Understanding Performance Interference of I/O Workload in Virtualized Cloud Environments

 

 

 

 

 

 

 

 

 

Benchmarking cloud serving systems with YCSB

 

 

 

 

 

 

 

 

 

Big data and cloud computing: new wine or just new bottles?

Dragomir Rares

243

12/06/2013

16:00

drbd0597

9

1;1

1

 

 

A self-organized, fault-tolerant and scalable replication scheme for cloud storage

 

 

 

 

 

 

 

 

 

Workload-aware live storage migration for clouds

 

 

 

 

 

 

 

 

 

Experiences on Processing Spatial Data with MapReduce

 

 

 

 

 

 

 

 

 

Cost of Virtual Machine Live Migration in Clouds: A Performance Evaluation

Velicea Livia Maria

244

29/05/2013

16:00

vlsd0664

9.5

5;1

10

 

 

Q-clouds: managing performance interference effects for QoS-aware clouds

 

 

 

 

 

 

 

 

 

RC2-a living lab for cloud computing

Oltean Dorin

244

 

odsd0658

 

3;7

10

 

 

Towards elastic transactional cloud storage with range query support

 

 

 

 

 

 

 

 

 

On securing untrusted clouds with cryptography

Popescu Oana

254

05/06/2013

posd0359

10

8;1

10

8

 

Transparent security for cloud

Mihut Horea Andrei

243

 

mhbd0609

8;1

5;1

1

 

 

Adaptive query execution for data management in the cloud

 

 

 

 

 

 

 

 

 

VMWARE Virtualization

Mikehazi Cristian

244

11/06/2013

16:00

Mcsd0655

9

9;1

10

 

 

On securing untrusted clouds with cryptography

Mahu Aleaxndru

244

 

Masd0650

 

8;1

10

 

 

User Provided Cloud COmputing: Eucalypt+OpenStack

Susanu Radu

243

11/06/2013

16:00

 

7

1;7

1

 

 

Grid Computing using Cloud

Visan Mihai

253

 

vmbd0461

6

8;1

1

6

 

Mira

Cantea Andrei

244

 

Casd0635

9

10;10

10

 

 

Blob Storage

Stoia Sebastian

243

12/06/2013

16:00

Ssbd0619

9

5.5;1

10