A Software Defined Scalable and Flexible Container Management

A Software Defined Scalable and Flexible Container Management

A Software Defined Scalable and Flexible Container Management Solution Bin Xu Yong Feng Li Cao 2016 IBM Corporation #ibmedge Please Note IBMs statements regarding its plans, directions, and intent are subject to change or withdrawal without notice and at IBMs sole discretion. Information regarding potential future products is intended to outline our general product direction and it should not be relied on in making a purchasing decision. The information mentioned regarding potential future products is not a commitment, promise, or legal obligation to deliver any material, code or functionality. Information about potential future products may not be incorporated into any contract. The development, release, and timing of any future features or functionality described for our products remains at our sole discretion. Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon many factors, including considerations such as the amount of multiprogramming in the users job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve results similar to those stated here. #ibmedge 2 Agenda Container is Leading the IT Transformation Container Introduction Container Ecosystem IBM Spectrum Computing Offerings for Container Management IBM Spectrum Conductor Family Capabilities Reference Cases Demo

#ibmedge 3 Container is Leading the IT Transformation Cloud is widely adopted by most of the companies. 95% companies are using cloud, and the adoption of private/hybrid cloud is steadily increasing. The interesting in Container(Docker) is expanding rapidly. Container: A group of processes run in isolation Uses cgroups to limit resource consumption Uses namespaces for isolation Container History Started on Unix with chroot in 1979. Variants in the evolution: Source: RightScale FreeBSD jails in 1998, SWsoft's (later Parallels') Virtuozzo in 2001 (and the associated OpenVZ in 2005), Solaris Zone in 2005, workload partitions (WPARs) on AIX and HP-UX Containers in 2007. LXC (2008): Modern Linux container, OS container Docker(2013): Build, Ship, run. Focusing on application container #ibmedge 4 Containers and VMs Solve Different Problems Containers is more light and better performance: Portability: VM (Gigabytes) vs. Container (Megabytes), VMs are constraint to Hypervisor and hardware-emulation App1

App2 App3 Bins/ Libs Bins/ Libs Bins/ Libs App1 App2 App3 OS OS OS Bins/ Libs Bins/ Libs Bins/ Libs Hypervisor Container Engine Host Operating System Host Operating System

Host Server Host Server Virtual Machines Containers Performance: Containers can boot and restart in seconds, compared to minutes for virtual machines. And no extra overhead of a hypervisor and guest OS makes containers consume less CPU and memory. Management cost: Each VM requires a full functional operating system, and then extra management for them. Great advantage to use containers in: DevOps Batch computing Lightweight PaaS Microservices #ibmedge 5 Next-Generation Application Development Requires Efficient Container Management Microservices Architecture: Decomposed into small pieces Loosely coupled Easier to scale development Improved fault isolation Each service can be developed and deployed independently Eliminates any long-term commitment to a technology stack #ibmedge 6

Container Management Platform is Emerging Container Management Platforms are also known as container managers, container orchestration engines (COEs), container orchestration platforms and containers-as-a-service (CaaS) platforms. It helps enterprises address the challenges of deploying containers. These platforms let administrators monitor, manage, secure, and scale containers similar to how they treat existing servers and virtual machines. Image Management (Appstore, registry) Security / Compliance Monitoring & Logging HA / Backup & Restore Auto-scaling / Upgrade Service Registration & Discovery Routing & Load Balancing Scheduling Bare Metal / Virtualization Environment On-prem / off-prem #ibmedge 7 Container Ecosystem Layer 5 CMP Tool Chain Layer 4 Orchestration

Layer 3 Container Engine Layer 2 Operating System Layer 1 Infrastructure #ibmedge 8 IBM Spectrum Computing Offerings for Container Management 2016 IBM Corporation #ibmedge Software Defined Infrastructure Portfolio High Performance Analytics (Low Latency Parallel) High Performance Computing (Batch, Serial, MPI, Workflow) Next Generation Application (Spark, Container) IBM Spectrum LSF IBM Spectrum Conductor

IBM Spectrum Symphony Workload Engines Resource Management IBM Spectrum Computing IBM Spectrum Scale Data & Storage Management IBM Spectrum Cluster Foundation (OpenStack/xCAT) Infrastructure Management Flash Tape Disk Power x86 Linux on z docker VM On-premise, On-cloud, Hybrid Infrastructure (heterogeneous distributed computing and storage environment) Support and Services

#ibmedge 10 Key Components for Container Management IBM Spectrum Conductor IBM Spectrum Conductor is an on premises platform for managing containerized applications that is based on the container orchestrator Kubernetes. Through the implementation of a robust resource manager, Apache Mesos, IBM Spectrum Conductor is able to manage resources for hybrid environments that contain both containerized and non-containerized applications as well as mixed hardware architecture. IBM Spectrum Scale IBM Spectrum Scale is software-defined storage for high performance, large scale workloads on-premises or in the cloud. Built on the award winning IBM General Parallel File System (GPFS), this scale-out storage solution provides file, object and integrated data analytics for all kinds of workloads. IBM Spectrum Cluster Foundation IBM Spectrum Cluster Foundation automates the creation of multiple scale-out environments on a shared infrastructure used by multiple teams. The software creates an agile environment for running both high-performance computing (HPC), analytics and containers. #ibmedge 11 IBM Container Management Platform Options IBM Container Service IBM Spectrum Conductor Bluemix Local Bluemix Public or Dedicated Ecosystem: Docker Data Center, Mesosphere DC/OS, Redhat OpenShift On-prem, customer

managed On-prem, IBM managed #ibmedge Off-prem, IBM managed Embrace Open Source Community Value IBM Value-add Docker Hub Registry holds a repository of 75000+ Docker images Lots of application integrated with Mesos Kubernetes enable micro-service architecture Customer Value Client unique registry available on premises Security readiness guidance via the Vulnerability Advisor Build-in applications of popular open source projects and IBM enterprise products in App Store Access to the images and application you require to deploy containers that meet your business needs and strategy Open-source, standardized, lightweight, self

sufficient container technology Balance workload between on-prem and off-prem Deployment choice with openPOWER and x86_64 Flexibility to choose on-prem and offprem or mix for your business Build, ship, and run standardized containers Integrated monitoring & logging Elasticity to grow storage & container needs Integrated CI/CD flow Life-cycle management of containers and data volumes Docker ease of use combined with enterprise-level integrity and confidence #ibmedge 13 Simplify Container Management with Integrated Controls App Catalog Integrated Unified UI Open Secure Highly Available Customermanaged Container

Orchestration IBM Spectrum Conductor Resource scheduling Auto-scaling Resource Management Security On-Premise Developm Developm ent ent Delivery Delivery Infrastruc Infrastruc ture ture #ibmedge Power & X86 14 Full Lifecycle Management for Container Container orchestration Resource management

Application life-cycle management/schedule/deployment Scaling, Rolling upgrade Service Registry/Discovery Unified Web-based Interface Load Balance Multi-host Networking Distribute storage management Image/Software repository management Configure management Logs/Meters/Alert User/Account management (Multiple tenancy & RBAC) Private PrivateRegistry Registry IBM Spectrum Conductor Container infrastructure Spark Pattern based Pattern based cluster template Computing Computing

cluster template Framework Framework Monitoring Monitoring and and Reporting Reporting Cloud Native Resource Management Ops management x86 Installation/upgrade Health check #ibmedge Hypervisor 15 15 IBM Spectrum Conductor Overview Architecture Containerized Applications App Store Authentication Authorization Persistent

Volume Service Discovery Service Load Balance Distributed Key-value Store Troubleshooting Monitor Network Topology Image registry Cognitive Spark and Big Data Analytics Workflow GUI Kubernete s Session Scheduler HPC Installer (Deploy, Reconfigure,

HA, Scale, Rolling update) Mesos Master Mesos Agent K8s executor pod pod pod #ibmedge container container container container 16 16 IBM Spectrum Conductor Architecture Master Node km ctrl manager Heapter master mgr Boot Node km scheduler

etcd Kube-DNS Ansible based installer and ops manager GUI km agent km proxy Pod Pod Pod Flanneld Flanneld Agent Node Keystone Agent Node #ibmedge LDAP Server VIP Mesos Agent km proxy km agent

Docker Docker Pod cfc-auth Mesos Agent Mesos Agent Pod MySQL Image-mgr VIP VIP km agent Mesos master cfc-router appstore network mgr haproxy km apiserver km proxy

Docker Pod Pod Pod Pod Flanneld Agent Node 17 17 IBM Spectrum Conductor GUI Improve Developer Productivity 1 Create a Container Cloud for developers supporting DevOps practices and cloudnative apps. Pre-built app catalog for fast deployment of OSS tools. Reduce developer friction, creating faster time to results Increase Resource Utilization 2 Fine grain, dynamic allocation of resources maximizes efficiency of Spark instances sharing a common resource pool. 3 Reduce Administration Costs Proven architecture at extreme scale, with enterprise class workload management, monitoring, reporting, and security capabilities. #ibmedge

| 18 18 Response to the Lesson Learned Gaps in current state of Kubernetes Single installer and operation manager to manage disparate parts together Unified GUI as management console for various services Single API end-point Single user service end-point and load balancer Central authentication and authorization manager Resource manager to support various workload manager and fine-grain resource sharing Enterprise Requirement HA topology System services live rolling upgrade and live reconfiguration Heterogonous environment (Power, X86, GPU and so on) Trouble-shooting, audit, alarm and event Multiple site #ibmedge 19 IBM Spectrum Conductor Community Edition Community Edition v0.1 (Tech Preview) is releasing soon! Free to use as you wish. We are looking for feedback for our roadmap. Register on our community page: http://ibm.biz/ConductorForContainers #ibmedge 20 Release Timeline Information regarding potential future products is intended to outline our general product direction and it should not be relied on in making a purchasing decision. The information mentioned regarding potential future products is not a commitment, promise, or legal obligation to deliver any material, code or functionality. Information about potential future products may not be incorporated into any contract. The development, release, and timing of any future features or functionality described for our products remains at our sole discretion.

Community Edition 0.1 Community Edition 0.1 Initial version Kubernetes&Mesos API/CLI Initial version GUI Kubernetes&Mesos API/CLI GUI Installer and HA Authentication Installer and HA LDAP App store LDAP Authentication App Private store image registry Sample Appsregistry in App Store Private image SampleNginx, AppsSockShop in App Store Build-in Network Nginx, SockShop Flannel Build-in Network Build-in Persistent Storage Flannel NFS, Glusterfs Build-in Persistent Storage HW Support

NFS, Glusterfs Power, x86 HW Support Community Edition 0.2 Community Edition 0.2 Spark Spark Spark Session Scheduler OpenSession SourceScheduler Spark Rebase on fr8r Open Source Kubernetes Rebase on fr8r Build-in ingress service Kubernetes HW Support Build-in ingress service Z HW Support Z Community Edition 0.3 Community Edition 0.3 CI/CD flow Jenkins CI/CD flow Jenkins git/cvs Jenkins Jenkins private docker registry Jenkins git/cvs Batch private docker registry Jenkins Batch Kubernetes batch

Build-in App in App Store Kubernetes batch Marathon, Tomcat, Build-in App in App StoreReact & Django, blockchain, Marathon, Tomcat, Reacttensorflow, & R-studio, OpenCV, Django, blockchain, tensorflow, R-studio, OpenCV, Power, x86 3Q16 4Q16 #ibmedge 1Q17 21 More challenges when setting up the environment onpremise Cluster Ops (Day-0, ) Repeatable automatic deployment with great customized flexibility Build on bare metal directly Complex hardware environment: (x86/Power, GPU, etc) Continually monitoring, maintenance and upgrade Hybrid services from Developer

Container service is major, but VM service (even bare-metal) is still required for special case. Multiple clusters Multiple regions Isolated clusters to address strong security Resource balancing #ibmedge 22 Software Define the Infrastructure with Templates Overview Powerful lifecycle management for scale-out cluster environments IBM Spectrum LSF Key Capabilities Benefits Faster time to cluster readiness Unified interface for management and monitoring Increased administrator productivity Single infrastructure supporting multiple business needs #ibmedge IBM Spectrum Cluster Foundation Unified Web-based Interface Cluster Clustertemplate template designer designer Simplified management with cluster template designer Scales from single clusters to complex multi-team environments

Robust, scalable alerting and reporting Automated infrastructure management one-click cluster deployment Enhanced Cluster management: cluster maintenance, health check and cluster upgrade (Bulk and Rolling) Pattern based Pattern based cluster template Workload based Workload based cluster template cluster template cluster template Monitoring Monitoring and and Reporting Reporting IBM Spectrum Conductor Infrastructure Management x86 Hypervisor 23

IBM Spectrum Conductor with Infrastructure Management DevOps PaaS Simplify IT operations BD & A 1 Discover bare metals and quickly deploy the environment on-demand (bare metal, virtualization or hybrid) Load Balance Authentication Image Registry (OS, VM, container) Increase Resource Utilization 2 Fine grain, dynamic allocation of resources maximizes efficiency of servers (Bare metals and VMs) sharing a common resource pool. SW Repository Logging/Metric Alert & Policy 3 Infrastructure Resource Aggregation xCAT Bare-Metal Generic Cluster

Deployment Public Cloud adapter #ibmedge Infrastructure discovery Reduce Administration Costs Proven architecture at extreme scale, with enterprise class infrastructure management, monitoring, reporting, and security capabilities. 24 Deliver an Agile Containerization Infrastructure in Enterprise Benefits Auto deploy customized OpenStack to offer the virtualization pools Auto deploy two container management environments on both bare metals and virtual machines. Easy to adjust the size of container management environments to balance the workload and full Building up Multi-tenant management based on LDAP IBM Spectrum Conductor Cluster#1 Design upgrade Automation Automation Deploy

POD Monitor & Health scale POD Virtualizations Pools IBM Spectrum Conductor Cluster#2 POD POD VM VM Docker Engine VM VM POD VM VM VM VM OpenStack (KVM) IBM Spectrum Scale Operating System POD IBM Spectrum Scale Elastic scale in/out

Operating System Bare BareMetal Metal Bare BareMetal Metal IBM IBMSpectrum SpectrumCluster ClusterFoundation Foundation Cluster Template xCAT Orchestration Orchestration OpenStack Provisioning Storage #ibmedge Network Server Server Server 25 Reference Cases 2016 IBM Corporation

#ibmedge Requirements from a China Telecom Customer Resource Management (Mesos) Support Non-Container Service (Marathon) Support Container Service (Kubernetes) Support Big Data Applications (Myriad vs. YARN on Kubernetes) Support Service Categories and Routers Unified DNS server for Service registration and discovery Support Software Defined Network Based on HA-proxy Support Application Level Resource Sharing and preemption

Provide resource sharing and preemption: When resources for Big Data are free, other application can borrow idle resource; and intelligent recovered when needed Support Infrastructure Level dynamic balancing Applications could be scheduled to bare-metal and VMs according to application type Bare-metal pool node and virtualization pool could be balanced #ibmedge | 27 27 Container Management Solution Architecture for Telecom Application Catalog Container Service Others Big Data Customer Solution k8sm-apiserver k8sm-scheduler Kubernetes k8sm-controllmanager Myriad Schedule

Marathon IBM Solution IBM Value-adds r Portal Auth Heterogeneous Nodes Resource Scheduler APP Store Mesos Agent Mesos Agent Mesos Agent Software Defined Network (Flannel, DNS) ... Mesos Agent Mesos Agent Software Defined Storage (Ceph, IBM Spectrum Scale) #ibmedge

| 28 28 Demo 2016 IBM Corporation #ibmedge Q&A 2016 IBM Corporation #ibmedge Notices and Disclaimers Copyright 2016 by International Business Machines Corporation (IBM). No part of this document may be reproduced or transmitted in any form without written permission from IBM. U.S. Government Users Restricted Rights - Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM. Information in these presentations (including information relating to products that have not yet been announced by IBM) has been reviewed for accuracy as of the date of initial publication and could include unintentional technical or typographical errors. IBM shall have no responsibility to update this information. THIS DOCUMENT IS DISTRIBUTED "AS IS" WITHOUT ANY WARRANTY, EITHER EXPRESS OR IMPLIED. IN NO EVENT SHALL IBM BE LIABLE FOR ANY DAMAGE ARISING FROM THE USE OF THIS INFORMATION, INCLUDING BUT NOT LIMITED TO, LOSS OF DATA, BUSINESS INTERRUPTION, LOSS OF PROFIT OR LOSS OF OPPORTUNITY. IBM products and services are warranted according to the terms and conditions of the agreements under which they are provided. IBM products are manufactured from new parts or new and used parts. In some cases, a product may not be new and may have been previously installed. Regardless, our warranty terms apply. Any statements regarding IBM's future direction, intent or product plans are subject to change or withdrawal without notice. Performance data contained herein was generally obtained in a controlled, isolated environments. Customer examples are presented as illustrations of how those customers have used IBM products and the results they may have achieved. Actual performance, cost, savings or other results in other operating environments may vary. References in this document to IBM products, programs, or services does not imply that IBM intends to make such products, programs or services available in all countries in which IBM operates or does business. Workshops, sessions and associated materials may have been prepared by independent session speakers, and do not necessarily reflect the views of IBM. All materials and discussions are provided for informational purposes only, and are neither intended to, nor shall constitute legal or other guidance or advice to any individual participant or their specific situation. It is the customers responsibility to insure its own compliance with legal requirements and to obtain advice of competent legal counsel as to the identification and interpretation of any relevant laws and regulatory requirements that may affect the customers business and any actions the customer may need to take to comply with such laws. IBM does not provide legal advice or represent or warrant that its services or products will ensure that the customer is in compliance with any law

#ibmedge 31 Notices and Disclaimers Cont. Information concerning non-IBM products was obtained from the suppliers of those products, their published announcements or other publicly available sources. IBM has not tested those products in connection with this publication and cannot confirm the accuracy of performance, compatibility or any other claims related to non-IBM products. Questions on the capabilities of non-IBM products should be addressed to the suppliers of those products. IBM does not warrant the quality of any third-party products, or the ability of any such third-party products to interoperate with IBMs products. IBM EXPRESSLY DISCLAIMS ALL WARRANTIES, EXPRESSED OR IMPLIED, INCLUDING BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. The provision of the information contained herein is not intended to, and does not, grant any right or license under any IBM patents, copyrights, trademarks or other intellectual property right. IBM, the IBM logo, ibm.com, Aspera, Bluemix, Blueworks Live, CICS, Clearcase, Cognos, DOORS, Emptoris, Enterprise Document Management System, FASP, FileNet, Global Business Services , Global Technology Services , IBM ExperienceOne, IBM SmartCloud, IBM Social Business, Information on Demand, ILOG, Maximo, MQIntegrator, MQSeries, Netcool, OMEGAMON, OpenPower, PureAnalytics, PureApplication, pureCluster, PureCoverage, PureData, PureExperience, PureFlex, pureQuery, pureScale, PureSystems, QRadar, Rational, Rhapsody, Smarter Commerce, SoDA, SPSS, Sterling Commerce, StoredIQ, Tealeaf, Tivoli, Trusteer, Unica, urban{code}, Watson, WebSphere, Worklight, X-Force and System z Z/OS, are trademarks of International Business Machines Corporation, registered in many jurisdictions worldwide. Other product and service names might be trademarks of IBM or other companies. A current list of IBM trademarks is available on the Web at "Copyright and trademark information" at: www.ibm.com/legal/copytrade.shtml. #ibmedge 32 Thank You 2016 IBM Corporation #ibmedge

Recently Viewed Presentations

  • Department of Health Standard Screen PowerPoint Template

    Department of Health Standard Screen PowerPoint Template

    Western Australian Multidisciplinary Post Fall Guidelines 2018 . A review of the Post Fall Management Guidelines in Western Australian Healthcare Settings 2015 was undertaken in 2018 by a large working party formed of health professionals from across public, rural, and...
  • Applying COSO's Enterprise Risk Management — Integrated Framework

    Applying COSO's Enterprise Risk Management — Integrated Framework

    Manajemen Risiko Investasi Suheri, Ir., M.Si Pertemuan Bond Fundamntal Fundamental of Probability Fundamntal of Statistics Bond Fundamental Manajemen Risiko diawali dengan Harga dari Aset selanjutnya potensi perubahan harga aset dengan berubahnya interest rate - Diskonting, PV, FV - Hubungan antara...
  • ICC - Paris - 2004

    ICC - Paris - 2004

    Pc = 7.38 MPa Low toxicity High stability Minima correspond to pore width with large distance between adjacent layers The disjoining force is repulsive when nanoparticles are close, then becomes attractive, and finally diminishes to zero when the separation is...
  • Youth Lacrosse Coaching Strategy, Tactical Thoughts ...

    Youth Lacrosse Coaching Strategy, Tactical Thoughts ...

    The attack man's responsibility is to score goals and control the offense. ... Scooping, catching and throwing mechanics on the run. ... Youth Lacrosse Coaching Strategy, Tactical Thoughts, Practical Tips and Drills
  • Reverse Engineering - University of Birmingham

    Reverse Engineering - University of Birmingham

    Reverse Engineering Ian Kayne For School of Computer Science, University of Birmingham 2nd February 2009 RCE Reverse (Code) Engineering - "reversing" What is it? Why is it done? Malware research & defence System interoperability requirements Review and audit of software/security...
  • Diapositive 1 - BALERDI

    Diapositive 1 - BALERDI

    L'heure moyenne de Greenwich (GMT pour UTC), longtemps calculée par l'Observatoire, fut à partir du XIXe siècle la référence des fuseaux horaires, avant d'être remplacée par le temps universel coordonné (UTC) qui a conservé pour origine ce fuseau. Autour de...
  • Acls Ch05

    Acls Ch05

    chapter 5 culture and health care
  • The Structure and Function of Macromolecules Part II:

    The Structure and Function of Macromolecules Part II:

    While none of these distinctions will be asked on the AP Biology exam, they certainly will on the AP Chemistry exam and should be taught in Chem I as well. It's not surprising that students are confused since the vocabulary...