Pivotal Cloud Foundry 1.6 ships with .NET support by Mark Kropf

Last week Pivotal shipped the 1.6 release of its enterprise distribution of Cloud Foundry (PCF). This release is a really big deal for a number of reasons such as: spring cloud services, diego, docker, and .NET. The Pivotal team in NYC was able to GA the .NET support delivered in PCF at the same time as the new Elastic Runtime (Diego) was integrated into PCF. This marks a major milestone for PCF customers by adding native support for .NET developers that leverages containers we've implemented on Windows 2012. 

Read More

General Availability of .NET Support on Windows for Cloud Foundry by Mark Kropf

After a busy 10 months of active development by Cloud Foundry foundation members from Centurylink, HP, and PivotalGarden-Windows & Diego-Windows are now GA'd. This first release targets Windows 2012 R2 and provides support for a broad set of .NET applications. The release of this .NET support on Windows corresponds with the GA of the Cloud Foundry Diego runtime architecture. This new runtime architecture allowed us to implement windows support natively within the common components of the Cloud Foundry runtime. 

Read More

Setup Lattice to run your Windows Apps by Mark Kropf

As we get closer to shipping Diego with the accompanying Windows support provided by the Greenhouse project, we've found Lattice useful for testing windows apps. To get a functioning windows cell, first you need to install a few pre-requisites and configure the instance.

Configuring a Windows Instance

The following script is what I use on AWS Windows 2012 Core instances, you may need additional configuration depending on your IaaS and Windows image.

dism /online /Enable-Feature /FeatureName:IIS-WebServer /All /NoRestart
dism /online /Enable-Feature /FeatureName:IIS-WebSockets /All /NoRestart
dism /online /Enable-Feature /FeatureName:Application-Server-WebServer-Support /FeatureName:AS-NET-Framework /All /NoRestart
dism /online /Enable-Feature /FeatureName:IIS-HostableWebCore /All /NoRestart

netsh interface ipv4 add dnsserver "Ethernet" address=127.0.0.1 index=1
netsh interface ipv4 add dnsserver "Ethernet" address=8.8.4.4 index=2
netsh interface ipv4 show dnsservers

sc config Dnscache start= disabled
sc stop Dnscache

:: enable disk quotas on C drive
fsutil quota enforce C:

In addition to the configuration above, you'll need to to increase the desktop heap for services and open firewall rules for all the diego services. 

Installing Diego components on Windows

After your windows instance is configured, you'll need to install & configure: Consul, Containerizer, Garden-windows, Executor, and Rep. We have built an unattended MSI installer that will do this for you which is currently in closed beta. Our plan is to open this beta installer up for download on Pivotal Network

The following configuration detail is needed to get the above components running ahead of the release of our MSI installer.

EXTERNAL_IP=[External IP of box]
CONSUL_IP=[Consul agent IP from the lattice brain]
ETCD_CLUSTER=[IP address of the ETCD server, often the lattice brain]
MACHINE_NAME=[This machine's name (must be unique across your cluster)]
STACK=[CF stack, eg. windows2012R2]
REDUNDANCY_ZONE=[Diego zone this cell is part of, eg. z1]
LOGGREGATOR_SHARED_SECRET=[loggregator secret from lattice config]

Building your Application for Lattice

1. Install Visual Studio & Nuget
2. Clone Nora from https://github.com/pivotal-cf-experimental/nora
3. Run /nora/assets/nora/make.bat from the cmd
4. Create a tgz of the /nora/assets/nora/Nora folder
5. Upload the tgz to a web location accessible to your lattice nodes.

Building the Diego Windows Lifecycle

To create a new Desired LRP in Lattice, you will need a windows lifecycle accessible to your lattice instances. To build a windows lifecycle, follow the following steps:

1. Clone https://github.com/cloudfoundry-incubator/windows_app_lifecycle
2. Run ./windows_app_lifecycle/make.bat in cmd
3. Upload resulting windows_app_lifecycle.tgz to a web location accessible to your lattice nodes.

Crafting the Desired LRP for your Windows Application

Since Lattice does not support windows apps natively, you must craft your own Desired LRP JSON file. Here is a template of one that works with our nora application:

{
    "setup": {
      "serial": {
        "actions": [{
          "download": {
            "from": "<INSERT WINDOWS LIFECYCLE URL>",
            "to": "/tmp/lifecycle",
            "cache_key": ""
          }
        }, {
          "download": {
            "from": "<INSERT YOUR APP URL>",
            "to": ".",
            "cache_key": ""
          }
        }]
      }
    },
    "action": {
      "run": {
        "path": "/tmp/lifecycle/launcher",
        "args": [
          "app",
          "",
          "{\"start_command\":\"tmp/lifecycle/WebAppServer.exe\",\"start_command_args\":[\".\"]}"
        ],
        "env": [{
          "name": "PROCESS_GUID",
          "value": "<INSERT APP NAME>"
        }, {
          "name": "PORT",
          "value": "8080"
        }],
        "resource_limits": {},
        "log_source": "APP"
      }
    },
    "monitor": {
      "timeout": {
        "action": {
          "run": {
            "path": "/tmp/lifecycle/healthcheck",
            "args": [
              "-port=8080"
            ],
            "env": null,
            "resource_limits": {},
            "log_source": "HEALTH"
          }
        },
        "timeout": 30000000000
      }
    },
    "process_guid": "<INSERT APP NAME>",
    "domain": "lattice",
    "rootfs": "preloaded:windows2012R2",
    "instances": 1,
    "env": [{
      "name": "PROCESS_GUID",
      "value": "<INSERT APP NAME>"
    }, {
      "name": "PORT",
      "value": "8080"
    }],
    "start_timeout": 60,
    "disk_mb": 1024,
    "memory_mb": 1024,
    "cpu_weight": 10,
    "privileged": true,
    "ports": [
      8080
    ],
    "routes": {
      "cf-router": [{
        "hostnames": [
          "<INSERT DESIRED APP LATTICE URL>"
        ],
        "port": 8080
      }]
    },
    "log_source": "APP",
    "log_guid": "<INSERT APP NAME>",
    "metrics_guid": "<INSERT APP NAME>",
  }

Fill in all the data captured in the '<>'s and save the file as noralrp.json. 

Create a new Lattice App of Nora

After installing the lattice cli run the following command: 

ltc create-lrp ./noralrp.json

This should give you a functional Windows cell running in Lattice with the Nora test ASP.NET application running. Please report any bugs you find when running your apps to the appropriate github repo. Most of the windows specific code resides in containerizer and it's dependent repos. 

Project Greenhouse Update by Mark Kropf

The Greenhouse Project team has reached another milestone towards our first release. The team recently completed the required garden api endpoints to support the initial Diego functionality. The project is tracking well toward a release with Diego coming up this summer. 

We've been impressed with the amazing support that Microsoft has provided us both on this project and the Azure CPI project for Bosh. We are eager to continue to partner with them to produce the most secure windows container available. There will be a great session this year at summit discussing their involvement with Cloud Foundry on the CPI project.

The Greenhouse team is now engaged with multiple Cloud Foundry Foundation members with exciting new Windows contributions to be announced at CF Summit on May 11th and 12th in the Windows CF ecosystem. 

Finally, I'd like to share a quick teaser video showing off the installer work the team has been working on. This installer along with a script to prep a fresh install of Windows Server 2012 for Diego will be available on PivNet soon. In the video you'll also see me push our test application Nora and scale it up while getting log data from our log aggregation system.

I'd also like to once again thank our amazing Greenhouse NYC dev team along with our new Greenhouse Product Manager Kimberly.

An application developer's view of the CF stack by Mark Kropf

Often when I'm talking to developers about Cloud Foundry, I get questions about where the different application dependencies come from. All applications deployed on Cloud Foundry rely on the following stack to provide it an operating system and needed middleware.

Stemcell

In this article I'm going to only focus on the role of the stemcell as it pertains to what a developer pushing an app on Cloud Foundry would see. The stemcell is responsible for a much larger portion of the stack on other components of CF and other bosh deployments.

As a developer, the stemcell is primarily responsible for providing the application a kernel and devices required for the application to run. The drivers for the network interface, semaphores required by compilers, and other kernel specific dependencies are also provided by the stemcell that is deployed to the DEA's (or in Diego lingo Cells). Today CF primarily uses an Ubuntu Trusty 14.04 stemcell, with support for RHEL7/CentOS7 and Windows Server 2012 coming soon.

RootFS

Every container that is spun up in Cloud Foundry receives an root filesystem from an image that is purpose built for CF use. This image is kept as minimal as possible, only providing the bare essential dependencies for a Cloud Foundry buildpack to be able to run. For this reason, change here is minimized to essential security patches only. Keeping this layer consistent ensures applications and buildpacks continue to function properly as the rest of the stack iterates. 

Buildpack

Buildpacks in Cloud Foundry are truly where the magic happens that enables developers to focus on developing business logic and enables ops folks to standardize their deployed stack. The format of a buildpack was adopted from Heroku and is being iterated to meet increasingly more robust application needs. Primarily buildpacks come with the following three scripts and any bundled dependencies required for deploying a user's application.

Detect

The detect script of each buildpack is executed in order of priority as defined by the CF operator. The first detect script that positively identifies the application as a match will end the detect process. Once a buildpack has detected an application it is capable of deploying, the compile process begins.

Compile

The compile script is responsible for providing the user's application with all the required middleware, app dependencies, and finally compiling a re-usable image of the end result. Often this involves installing a framework and/or web server along with running any staging commands against the application framework to download dependencies (i.e. maven, npm, pip, gem, nuget, etc). The end result of this process is an image of the application deployed and staged against the rootfs + stemcell kernel. This image called a droplet is then stored in the Cloud Foundry blobstore for immediate use.

Release

Finally the release script provides execution metadata back to Cloud Foundry about how to execute the application and what environment variables need to be set for the application at execution. 

Use Patterns

It is a common practice for a CF operator to constrain the middleware/framework versions provided by a buildpack to establish a standard stack within an enterprise setting. It is also possible for developers to bring their own custom buildpack along with their application to provide a custom stack for their application not found in any of the default buildpacks. It is very easy for change to be made at this level of the stack without updating the entire environment. This is an ideal layer for customization and iteration of application dependencies. 

Summary

Cloud Foundry leverages this unique layering of the traditional application host stack to enable devs to refocus on developing business value. Operators are given the needed tooling to establish a consistent execution environment while maintaining a very ephemeral (cattle) view of their deployed applications.

REPOS for Windows Support in Cloud Foundry now public by Mark Kropf

Today the collaborative effort between Pivotal and CenturyLink has reached another milestone in adding support for windows in Cloud Foundry. After approval from the Cloud Foundry CAB we've promoted the repos needed to implement windows support in diego-release to the CloudFoundry Incubator org on github. This will allow the Cloud Foundry community a very early look into our work. The following repos are now publicly available and ready for contributions from the CF OSS community:

Repos

1. windows_app_lifecycle - The windows application lifecycle implements the traditional Cloud Foundry deployment strategy. The Builder downloads buildpacks and app bits, and produces a droplet. The Launcher runs the start command using a standard rootfs and environment. The Healthcheck runs a tcp port check, defaulting to port 8080. Read about the app lifecycle spec here: https://github.com/cloudfoundry-incubator/diego-design-notes#app-lifecycles

2. garden-windows - This is a Windows backend for garden. https://github.com/cloudfoundry-incubator/garden-windows

3. containerizer - Our soon to be renamed C# project that makes the windows kernel calls to containerize your application. https://github.com/cloudfoundry-incubator/containerizer

4. diego-release - We have a branch on the diego-release repo where we keep any updates needed to run a windows stack in diego. https://github.com/cloudfoundry-incubator/diego-release/tree/greenhaus

What's next?

You can always follow along with the team's progress using our tracker. The next big milestone for the team is to produce a beta release of the windows support via an msi installer. The beta will be the first time where a user could install this into an existing diego environment with minimal manual effort. Today the repos alone could be deployed to an environment with success, but with a lot of manual setup and hand holding. To be absolutely clear, this is not a alpha/beta release, just open sourcing the repos so we can start actively collaborating with the CF OSS community. 

Thanks

We're very fortunate to have such a talented team of developers from both CenturyLink and Pivotal contributing to this project. 

Thank you: Mike Dalessio, David Varvel, Dave GoddardDavid Morhovich, Brannon Jones, Bob Brumfield, Chris Sterling, Matt Cholick, and Jared Gordon

Stay tuned for more updates on our progress towards Windows support in CF

Spooky Controlling lights spookily fast (spukhafte Fernwirkung) using nodejs by Mark Kropf

For the past few years I've been using Hexler's Touch OSC app to control lights driven from a variety of micro controllers (arduino, netduino, etc). Earlier this year I started playing with nodejs to drive my lights via a beaglebone black or raspberry pi

So far Spooky supports a variety of Chauvet and Eliminator DMX fixtures via an ArtNet to DMX gateway. I use the EntTec ODE model, but any ArtNet to DMX gateway should work. Due to my inability to get NodeJS to speak SPI to my hanging lantern lights, I only can control those via having Spooky speak OSC to my ruby app that controls the octobars. I'm looking at having node send instructions to another C app that will drive the spi in the future. MS accurate timing in a do everything async framework is a bit difficult, and probably isn't a problem NodeJS is looking to solve.

All fixtures in Spooky are controlled via Open Sound Control ( OSC ). I've included a few TouchOSC templates in the repo for those looking for a quick start. Spooky does not currently support responding to clients with an updated control state. I'm looking at refactoring Spooky to update a state machine of every lighting fixture on every inbound request to enable responding to connected clients with the state of every control.

In addition to lighting fixtures, I've added support for the Chauvet Hazer I use for parties. I've done my best to make adding new devices to spooky as easy as possible. 

Spooky Github: https://github.com/MarkKropf/spooky

Here are a few of the TouchOSC layouts that are included:

colorado72.png
spooky_touchosc.png


The DrinkTime App by Mark Kropf

As an exercise to begin learning Swift (not the blobstore) I've started work on an app called DrinkTime. The idea for the application came from talking to my good bud Zach Robinson about how silly the Yo app is, and how they really missed the primary use of a single word communication app, Drinks?. The DrinkTime application is designed to ask the one word question everyone begins to think about as the end of the work day approaches. The DrinkTime app is broken down into 3 easy steps.

  1. Where do you propose having the drinks? Enter in a location or point of interest of the drinking establishment of your choice. It only takes one decisive person in the group to make this mission a success.
  2. Who should you prop the question of "Drinks?" to? Pick from a list of friends you've already added to the application.
  3. Send the message and watch as people update their status on who will be coming out to drink that night.

I'm hoping to learn a bunch with this app, and maybe even use it for after work drinks myself. Here's the list of hurdles I need to overcome:

  • Nobody likes creating yet another account, so I should partner with the least intrusive, most likely to have coworkers willing to share social platform. 
  • I'm not interested in recreating foursquare or untapped here, let's start out by pulling location data from someone that has it. Yelp?
  • Oh yeah, probably should learn enough of swift to be able to hack this together.
  • An app as epic as this should get its own set of artwork.

Ok, that about does it, hopefully there will be more posts showing how well this app is coming together. For now there's just this:

DrinkTime Alpha Screenshot 07/21/2014

DrinkTime Alpha Screenshot 07/21/2014