Hello!
Thank you for having me!
My name is Rich Jones!
Founder of gun.io!
(Awesome freelance gigs for F/OSS hackers!)
Author of
(shameless plug alert)
Zappa!
Python's serverless framework!
And..
Welcome to..
Serverless!: Lambda.. and Beyond!
⚠️⚠⚠️️️️️Warning!⚠️⚠⚠
This talk is going to move fast!
Better to be overwhelmed than bored!
Feel free to interrupt!
Ask me questions!
"What does serverless mean?!"
¯\_(ツ)_/¯
Part 1: Going Serverless With Zappa
aka: "no permanent infrastructure"
aka: "no permanent infrastructure"*
AWS Lambda + AWS API Gateway
There still are servers, but they are ephemeral
Traditional web server:
Zappa:
~30ms!
By the time the user sees the page, the server has disappeared.
ॐ
Advantages!
Super scalable!
1 request = 1 server
10 request = 10 server
100 requests = 100 servers
1000 requests = 1000 servers
10000 requests = 10000 servers
Orders of magnitude less expensive!
$0.000000002/ms
SGD$0.00000000275/ms
Plus 3,200,000 free seconds!
VPS: 4 * $20 * 12 = $960/yr
Zappa: $0.75 * 12 - Free = $0/yr
Zero maintainance!
Zero load balancing!
Zero security patches!
Zero downtime!
(fire your ops team)
"Cool! What else can it do?"
Event-driven architecture!
Code executes in response to events
File uploads!
Incoming emails!
New users!
"Neat! But I demand even more features!"
You're in luck!
Rollback! 
$ zappa rollback -n 2
Free SSL certificates! 
$ zappa certify
Log tailing! 
$ zappa tail
Auto keep-warm!
Use C-Extensions via lambda-packages!
Load remote environment variables from S3! 
// my-config-bucket/super-secret-config.json
{
    "DB_CONNECTION_STRING": "super-secret:database"
}

// zappa_settings.json
"remote_env_bucket": "my-config-bucket",
"remote_env_file": "super-secret-config.json"

# your_zappa_app.py
import os
db_string = os.environ('DB_CONNECTION_STRING')
CI/CD integration! 
$ zappa status dev --json
Remote command invocation! 
$ zappa invoke 'my_app.my_func'

$ zappa invoke 'print hello.upper()' --raw
Django management commands! 
$ zappa manage makemigrations
Don't need to modify your existing apps!
No vendor lock-in!
Battle tested!
Used in production by banks, governments, medical companies and more!
Works with any WSGI application!
Django!
(Wagtail! django-cms! Pinax!)
Flask!
Pyramid!
Bottle!
Hug!
"Wow! Okay! How do I get started!"
It's super easy! 
$ pip install zappa
$ zappa init


███████╗ █████╗ ██████╗ ██████╗  █████╗
╚══███╔╝██╔══██╗██╔══██╗██╔══██╗██╔══██╗
  ███╔╝ ███████║██████╔╝██████╔╝███████║
 ███╔╝  ██╔══██║██╔═══╝ ██╔═══╝ ██╔══██║
███████╗██║  ██║██║     ██║     ██║  ██║
╚══════╝╚═╝  ╚═╝╚═╝     ╚═╝     ╚═╝  ╚═╝


Welcome to Zappa!

Zappa is a system for running server-less Python web applications on AWS Lambda and AWS API Gateway.
This `init` command will help you create and configure your new Zappa deployment.
Let's get started!

Your Zappa configuration can support multiple production environments, like 'dev', 'staging', and 'production'.
What do you want to call this environment (default 'dev'):

  {
      "dev": {
          "aws_region": "us-east-1",
          "django_settings": "demo.settings",
          "profile_name": "default",
          "s3_bucket": "my-demo-bucket"
      }
  }
  
$ zappa deploy
Calling deploy for stage dev..
Downloading and installing dependencies..
Packaging project as zip..
Uploading demo1-dev-1502298757.zip (11.1MiB)..
Scheduling..
Scheduled demo1-dev-zappa-keep-warm-handler.keep_warm_callback with expression rate(4 minutes)!
Uploading demo1-dev-template-1502298785.json (1.6KiB)..
Waiting for stack demo1-dev to create (this can take a bit)..
Deploying API Gateway..
Deployment complete!: https://8x24a8pmn7.execute-api.us-east-1.amazonaws.com/dev
BAM!
You're server-less!
🐘
Database!
*uh-oh*
This is tricky!
Do you need a database?
Think serverlessly!
Avoid infrastructure!
ex: NoDB - incredibly simple, SQL-less, S3-based "database" 
$ pip install nodb
from nodb import NoDB

nodb = NoDB()
nodb.bucket = "my-s3-bucket"
nodb.index = "name"

# Save an object!
user = {"name": "Jeff", "age": 19}
nodb.save(user)

# Load our object!
user = nodb.load("Jeff")
print user.age # 19

# Delete our object
nodb.delete("Jeff") # True
ex: zappa-bittorrent-tracker can serve millions of peers with S3-database
Event-driven architecture!
This is the fun part!
Execute code in response to AWS ecosystem events!
No blocking pages!
No Celery!
Example 1: Avatar resizing
User uploads an image, we make a thumbnail
HTTP -> S3 -> Lambda -> S3 
// zappa_settings.yml
events:
- function: users.util.process_avatar
  event_source:
    arn: arn:aws:s3:::your-upload-bucket
    events:
    - s3:ObjectCreated:*

 $ zappa schedule
That's it! :D
*pro tip!*
Make sure you don't get stuck in an infinite loop! 
// zappa_settings.yml
events:
- function: users.util.process_avatar
  event_source:
    arn: arn:aws:s3:::your-upload-bucket
    events:
    - s3:ObjectCreated:/uploads/*
Example 2: Daily notifications
Time is an event source!
Send daily notifications to your Slack channel! 
// zappa_settings.yml
events:
- function: my_app.util.send_notifications
  expression: rate(24 hours)

 $ zappa schedule
Hooray!
🎉 🎉 🎉 🎉
"Sweet! But what if I don't want to wait for an event?
Zappa can execute functions asynchronously in a different Lambda!
Example 3: Let's bake a cake!
🍰 
def bake_cake(*args, **kwargs):
     ingredients = get_ingredients()
     cake = bake(ingredients)
     deliver(cake)

from zappa.asynchronous import task

@task
def bake_cake(*args, **kwargs):
     ingredients = get_ingredients()
     cake = bake(ingredients)
     deliver(cake)

# url('api/order/cake')
def order_cake(request):
    bake_cake()
    return HttpResponse("Your cake is being made!")
It's that easy!
No config!
No queues!
No Celery!
Go nuts!
🍰🍰🍰🍰🍰🍰🍰🍰🍰🍰🍰🍰🍰🍰🍰🍰🍰🍰🍰🍰🍰
"But what if I need to bake mission-critical cakes for the whole planet?"
Globally-available server-less architecture!
"That sounds cool! But why do I want that?"
If your servers are no-where, then they can be everywhere!
Benefits of global deployments:
#1: Redunancy!
Cloud computing is an act of faith.
Amazon goes down too!
*gasp*
Amazon goes down too..
..but (usually) not across the planet.
Pro tip!
Don't host your status page on your own infrastructure!
fail
#2: Speed! 
$ ping apigateway.us-east-2.amazonaws.com # Ohio
--- apigateway.us-east-2.amazonaws.com ping statistics ---
round-trip min/avg/max/stddev = 35.274/39.257/42.731/2.499 ms

$ ping apigateway.ap-northeast-1.amazonaws.com # Tokyo
--- apigateway.ap-northeast-1.amazonaws.com ping statistics ---
round-trip min/avg/max/stddev = 177.616/202.745/305.468/46.281 ms
>200ms just for the round trip!
(Earth is big!)
Please provide non-US and US users with equally great service!
#3: Scalability!
1 region: ~5,000 simultaneous connections
9 regions: ~45,000 simultaneous connections!
1,420,092,000,000 hits/year!
(Trillion! With a t!)
#4: Security!
Compartmentalize data, limit damage.
Defend against internal and external threats!
Prevent non-US employees accessing US data, etc.
PSA: This is a big topic. I'm not saying deploying globally makes you more secure by default. Consult your local hacker for more details.
#5: Regulatory compliance!
Different countries have different laws.
Medical Data
Financial Data
Personally Identifying Information
Private Communications
Data Retention
Etcetera..
(I'm not your lawyer. Thank goodness.)
One product may have different compliance needs in many countries.
"Well, I'm convinced! How do we do it?" 
$ zappa init

Do you want to make this a global application? Y

{
    "dev_ap_northeast_1": {
        "app_function": "geocake.app",
        "aws_region": "ap-northeast-1",
        "domain": "dev-ap-northeast-1.geocake.biz",
        "s3_bucket": "lmbda_dev_ap_northeast_1"
    },
    "dev_ap_northeast_2": {
        "app_function": "geocake.app",
        "aws_region": "ap-northeast-2",
        "domain": "dev-ap-northeast-2.geocake.biz",
        "s3_bucket": "lmbda_dev_ap_northeast_2"
    },

    [ ... ],

    "dev_us_west_2": {
        "app_function": "geocake.app",
        "aws_region": "us-west-2",
        "domain": "dev-us-west-2.geocake.biz",
        "s3_bucket": "lmbda_dev_us_west_2"
    }
}

$ zappa deploy --all
$ zappa certify --all
Bam! 
httpstat https://dev-ap-northeast-1.geocake.biz/
httpstat https://dev-ap-northeast-2.geocake.biz/
...
httpstat https://dev-us-west-2.geocake.biz/
RAWK \m/
Since we're at PyConSG, I'd like to talk about
Elastic BIG DATA processing!
This trick will be useful if you need to process a lot of data but don't want to manage infrastructure!
OR
You need to do a lot of data processing as part of a production system!
The "Fanout" Pattern!
One Lambda ->
Many Lambda ->
One Lambda
Rapid elastic parallel processing, no supercomputer required!
Let's see some examples!
Example 1: Parallel Data Processing
Suppose I have a large genetic dataset, and I want to process it on a sample-wise basis!
First, grab your dataset from refine.bio! 
from pandas import DataFrame
from my_fancy_models import classify, store_result
from zappa.asynchronous import task

def load_samples(sample_path="mysamples.tsv"):
  df = DataFrame.from_csv(sample_path, sep="\t")
samples = load_samples()

@task
def classify_sample(sample):
    classification = classify(sample)
    store_result(classification, sample)

def process_all_samples():
  for sample in samples:
    process_sample(sample)

$ zappa deploy
$ zappa invoke 'mybioapp.process_all_samples'
Done!
This code still works locally!
Mix local and remote code: 
from pandas import DataFrame
from my_fancy_models import classify, store_result
from zappa.asynchronous import task

def load_samples(sample_path="mysamples.tsv"):
  df = DataFrame.from_csv(sample_path, sep="\t")
samples = load_samples()

@task(
  remote_aws_lambda_function_name='my-classifier',
  remote_aws_region='us-east-1'
)
def classify_sample(sample):
    classification = classify(sample)
    store_result(classification, sample)

def process_all_samples():
  for sample in samples:
    process_sample(sample)
Example 2: ML in Production
Suppose you have lots of predictors, and you want to choose the one with the highest confidence.
If each of the predictors can operate in less than the space of a request, but not all together: 
import concurrent.futures
from zappa.asynchronous import run

predictor_functions = [
  'predictor.alpha',
  'predictor.beta',
  'predictor.delta'
]

def run_predictions(request):
  executor = concurrent.futures.ThreadPoolExecutor()
  results = executor.map(run, predictor_functions)

  highest_confidence = 0.0
  final_prediction = None
  for result in results:
      if result.confidence > highest_confidence:
        final_prediction = result.prediction
        highest_confience = result.confidence

  return final_prediction
One request -> Lots of predictions -> one response!
Hooray!
Hooray.. but!
My goal is to promote user freedom!
I just wanted a super easy and cheap way to host scalable services!
But now I've convinced thousands of people to use Amazon.
>:[
Part 2: Why You Shouldn't Use Zappa
Amazon has been repeatedly hostile to our community.
Because of AWS' hostility to our community,
we lost MongoDB, Redis and ElasticSearch to proprietary licenses!
>>:[
They did it to me!
>>>:[
They cloned my project, made it worse, didn't even give a link back.
"Embrace, Extend, Extinguish" all over again!
..and that's just AWS!
Amazon doesn't pay taxes,
destroyed small-town America,
overworks and underpays their employees,
all to further enrich the richest man in the world!
(Jeff Bezos is a bond villain)
Screw Amazon!
(I hope they're not a sponsor)
Let's build our own Lambda!
"LAMP"
"LAMP"
"LAMP" "PLONK"!
This gives us everything we need to build a free and open source alternative to AWS Lambda + API Gateway!
Run on any cloud host, or on your own hardware, or both at the same time!
Let's try it out!
It's as easy as..
(okay that's not super easy) 
$ faas
  ___                   _____           ____
 / _ \ _ __   ___ _ __ |  ___|_ _  __ _/ ___|
| | | | '_ \ / _ \ '_ \| |_ / _` |/ _` \___ \
| |_| | |_) |  __/ | | |  _| (_| | (_| |___) |
 \___/| .__/ \___|_| |_|_|  \__,_|\__,_|____/
      |_|


Manage your OpenFaaS functions from the command line

$ kubectl --namespace=openfaas get deployments -l "release=openfaas, app=openfaas"
NAME           READY   UP-TO-DATE   AVAILABLE   AGE
alertmanager   1/1     1            1           34m
faas-idler     1/1     1            1           34m
gateway        1/1     1            1           34m
nats           1/1     1            1           34m
prometheus     1/1     1            1           34m
queue-worker   1/1     1            1           34m

$ kubectl --namespace=openfaas get deployments -l "release=openfaas, app=openfaas"
NAME           READY   UP-TO-DATE   AVAILABLE   AGE
alertmanager   1/1     1            1           34m
faas-idler     1/1     1            1           34m
gateway        1/1     1            1           34m
nats           1/1     1            1           34m
prometheus     1/1     1            1           34m
queue-worker   1/1     1            1           34m
Let's make a function! 
$ faas new --lang python3 echoupper --prefix richjones
Function created in folder: echoupper
Stack file written: echoupper.yml

#echoupper.yml
version: 1.0
provider:
  name: openfaas
  gateway: http://localhost:8080
functions:
  sample:
    lang: python3
    handler: ./echoupper
    image: richjones/echoupper:latest

def handle(req):
     return req.upper()

$ faas up -f echoupper.yml

$ echo Hi! | faas-cli invoke echoupper
HI!

$ curl -d "Hi!" http://127.0.0.1:8080/function/echoupper
HI!
Sweet!
An extremely complicated, highly scalable, non-AWS echo cloud function!
(oh wow how useful :|)
More functionality in the OpenFaaS Function Store!
A growing collection of useful, community-contributed, free and open-source microservices for your application!
LEGO for big data processing and machine learning!
(oh, actually cool!)
!!! NEW PROJECT ALERT !!!
Fashion: Easy and Awesome Pythonic OpenFaaS!
Use OpenFaaS functions as if they were native,
but they run in their own containers! 
from fashion import figlet
hi = figlet("Hello, world!")
#  _   _      _ _                             _     _ _
# | | | | ___| | | ___    __      _____  _ __| | __| | |
# | |_| |/ _ \ | |/ _ \   \ \ /\ / / _ \| '__| |/ _` | |
# |  _  |  __/ | | (_) |   \ V  V / (_) | |  | | (_| |_|
# |_| |_|\___|_|_|\___( )   \_/\_/ \___/|_|  |_|\__,_(_)
#                    |/
No function definition required!
thanks, PEP562!
Chain functions together! 
from fashion import leftpad, figlet
figlet(leftpad("Hello!"))
#  _                _   _      _ _       _ _
# ( )              | | | | ___| | | ___ | ( )
# |/               | |_| |/ _ \ | |/ _ \| |/
#                  |  _  |  __/ | | (_) |_|
#                  |_| |_|\___|_|_|\___/(_)
Make asynchronous calls! 
from fashion import async_figlet
async_figlet("Hello!")
# "6a3ae7fb-a8ee-4988-b7de-e3b81d1aed65"
Chain async calls together to build ETL pipelines! 
from fashion import async_leftpad
cf = ["figlet", "send_email"]
async_leftpad("Hello", callback_functions=cf)
# "6a3ae7fb-a8ee-4988-b7de-d3b81d1aed65"
Limit functions to certain instance types!
(this is my favorite part)
Run the hard problems on expensive hardware, run the easy problems on cheap hardware! 
#update_model.yml
  [..]
   constraints:
     - "node.labels.instance_type == gpu"

# Runs on expensive GPU instance
from fashion import update_model

# Runs on cheap CPU instance
from fashion import send_result_email

result = update_model(input)
send_email(result)
# Email sent!
PLONK handles the placement and elasticity!
Automatic cost optimization at the application level!
✨✨✨magic!✨✨✨
Is this useful?
I don't know!
I hope so!
OpenFaaS is still young, but it is an exemplary open source project with a good team behind it!
Now we can use "serverless" patterns on our own hardware, or more ethical web hosts!
We're "serverless"..
We're "serverless".. but we have a lot of servers.
We're "serverless".. but we have a lot of servers. And tooling.
We're "serverless".. but we have a lot of servers. And tooling. And voodoo.
Goal: destroy ops.
Result: more ops!
>>>:[[[[
PLONK is good solution*..
..if you can handle the complexity..
..and you're already commited to Kubernetes..
..and you can afford the ops burden.
(Kubernetes: great for consultants, lousy for lazy hackers)
Fortunately, more cloud vendors offer off-the shelf Kubernetes, and dedicated OpenFaaS hosts are starting to appear.
But..
I'm just a regular guy!
I just want to run a service and not worry about servers!
Hmmm..
Okay then!
Let's go actually serverless!
As in, literally no servers!
For real this time!
Part 3: Let's build a new internet!
Enter..
The Peer-To-Peer Web!
There is an exciting landscape of new P2P technologies and protocols!
Solid, NextCloud, ZeroNet, GUN, IPFS, dat, Scuttlebot, DataShards, WebTorrent, and many more!
"RE-DECENTRALIZE"
This is what Tim Berners Lee is working on!
Less centralized corporate control!
Better global distribution and interconnectivity!
New models of privacy!
Better network resiliance!
Less censorship, more creative expression!
More freedom for users!
I want to run web services without any centralized servers!
Publish a service,
seed it,
peers visit,
peers keep seeding it
peers keep seeding it, even after the original seed goes away!
"Sounds cool, but that sounds impossible!"
We can do it right now!
Solid, GUN, IPFS, dat, DataShards, WebTorrent, and many more!
git + BitTorrent + HTTP = dat
git + BitTorrent + HTTP + ✨✨✨magic✨✨✨ = dat
Designed for data scientists (like you!) to share large, mutable datasets!
git is bad for data, dat is great for data!
Also works great with dynamic web content!
Let's publish a datsite! 
$ npm install dat
$ cd ~/src/my-website
$ dat .

$ dat .

Created new dat in /Users/rjones/pyconsg-datsite/.dat
dat://c81299cfc139791ccc6db42f0bcac8a9af590c03828e8066478fcfc60ca6e481
Super easy!
Users download the site each other!
Dats can link to dats, and dats can be updated, so we can build dynamic server-less applications!
This could be the begining of a new decentralized web!
It's also really fun, like the old web was!
We can replace Google, Twitter, Facebook, Netflix, Uber, Slack, Spotify, SoundCloud..
We can should replace Google, Twitter, Facebook, Netflix, Uber, Slack, Spotify, SoundCloud..
We can should must replace Google, Twitter, Facebook, Netflix, Uber, Slack, Spotify, SoundCloud..
with free and open source versions that don't spy on us!
I want to live in a post-advertising world!
"But they're too entrenched and powerful!"
(if you get that joke, you're officially old)
Dogpile, MySpace, Yahoo!, AOL, Snapchat, Digg, SourceForge, etc., etc.,
All tech incumbants can be dethroned!
sorry
(not sorry)
The opportunities are too large and too exciting to ignore!
Even better, we can build new types of collaborative services we couldn't imagine before!
A thousand projects on the magnitude of Wikipedia!
Free digital libraries for everybody in the world!
Rather than dismantling oppressive systems,
we can build liberated alternatives that make the old systems obselete!
This has been our story so far,
and it should be our story into the future as well!
There is so much to be done!
New opportunities come with new challenges, challenges are fun!
Next time you start a project, consider opportunities to redecentralize!
Okay!
In conclusion!
Save time and money,
build awesome event-driven applications,
never have downtime,
be fast everywhere,
use Zappa!
(your favorite companies are!)
Or,
if you don't want to give any more money to Jeff Bezos,
who is literally an evil Bond villain,
use PLONK and Fashion to run efficient data processing pipelines on your own FaaS!
(but you'll have to learn Kubernetes!)
(which you can do at the workshop tomorrow!)
Finally,
if you want to be part of the next generation of web technologies and services
join me in redecentralizing on the truly serverless peer to peer web!
(I have big plans!)
Want to contribute? 😃
100+ contributors,
6 continents!
🐧
Ways you can help:
Join the Slack! https://slack.zappa.io
Report bugs!
Fix bugs!
Pull request triage!
Build cool stuff!
Do you need awesome, scalable, server-less apps?
Hire me! 💸💸💸
Email: miserlou@gmail.com
Code: https://github.com/Miserlou/Zappa
https://github.com/Miserlou/Fashion
https://github.com/Miserlou/Talks
Thank you!
HUGE round of applause for our awesome organizers!
Questions?
🍺🍺🍺 Let's get some beers! 🍺🍺🍺