Google Cloud Platform enhances Compute Engine, App Engine and Cloud Datastore

By Greg DeMichillie, Google Cloud Platform team

Cross-posted from the Google Cloud Platform blog

Today’s updates to Google Cloud Platform bring lots of new enhancements complementing features we first shared at I/O. Check out the latest additions:

• layer 3 load balancing capability for Google Compute Engine
• new features and productivity improvements for Google Cloud Datastore
• improvements to the PHP runtime as part of the Google App Engine 1.8.3 release

Load balancing now available on Google Compute Engine
We’ve now added layer 3 load balancing to Compute Engine, which delivers Google-scale throughput and fault tolerance to manage Internet applications. Load balancing is critical in any highly scalable system, allowing you to automatically and intelligently route traffic across a collection of servers.

With the load balancing service, you can:

• Load-balance ingress network TCP/UDP traffic over a specific set of Compute Engine virtual machines (VMs) within the same region
• Ensure that only healthy VMs are used to serve Internet requests through the use of HTTP-based health checks
• Easily handle spikes in load without pre-warming
• Configure the load balancer via command line interface (CLI) and a programmatic RESTful API

This initial release provides Layer 3 support and we’ll continue to expand its capabilities on a regular basis. We’re pleased to offer the load balancing feature at no cost through the end of 2013, after which we’ll charge the rates outlined on our pricing page.

Improved developer experience for Google Cloud Datastore
Cloud development tools should enhance developer productivity, and that’s what we focused on with the latest update to Cloud Datastore.

Google Query Language (GQL) Support
Being able to search for data lies at the heart of all data-driven applications, and we’ve made that easier by adding GQL support — a SQL-like language for retrieving entities or keys from Cloud Datastore.

Metadata queries
You can now access Metadata Queries that allow you to retrieve statistics on their underlying data. This is useful when building your internal administration consoles, performing custom analysis or simply debugging an application.

Local SDK improvements
Part of providing great developer experiences is allowing developers to make changes in a fast, efficient and cost-effective way. That’s why we’ve made numerous enhancements to the command line tool. Separately, we’ve also included support for those of you who use Microsoft Windows.

Getting started with Ruby
If there is one thing that developers are passionate about (us included) it’s languages. The initial release of Cloud Datastore included code snippets and samples for getting up and running with Java, Python and Node. With the latest release, we’ve included the same Cloud Datastore support for Ruby developers — allowing you to easily spin up Ruby applications that rely on a managed NoSQL datastore.

Updates to the PHP runtime in Google App Engine 1.8.3
We’ve been listening to feedback from early developers using the PHP runtime, so our latest App Engine release is dedicated to PHP. But there are still a few new goodies for those of you not using PHP, which you can find in the release notes.

Our integration with Google Cloud Storage for reading and writing files has proven to be popular, so we expanded this in 1.8.3. Our latest update includes:

• improved support for working with directories in Google Cloud Storage — allowing you to call functions such as opendir() or writedir() directly on Cloud Storage buckets
• support for functions related to stat()-ing files, such as is_readable() and is_file()
• the ability to write metadata to Cloud Storage files
• substantial performance improvements through memcache-backed optimistic read caching — this improves the performance of applications that need to read frequently from the same Cloud Storage file
• numerous bug fixes

We’ve also improved support for task queues, including the ability to set headers on push tasks and to add tasks to queues efficiently in bulk using the new PushTask:addTasks() method.

We hope you enjoy the new Cloud Platform features!

Greg DeMichillie has spent his entire career working on developer platforms for web, mobile, and the cloud. He started as a software engineer before making the jump to Product Management. When not coding, he's an avid photographer and gadget geek.

Posted by Ashleigh Rentz, Editor Emerita

Getting started with Twilio on Google App Engine for PHP

This guest post was written by Keith Casey, Sr. Developer Evangelist for Twilio

Cross-posted from the Google Cloud Platform Blog

I’ve wanted to explore Google App Engine for years. Between its SLA, automatic scaling, and queuing system, it has always been compelling. Unfortunately, since my Python skills are somewhere between “Hello World” and “OMG What did I just do!?” I decided to save myself the embarrassment. When Google announced PHP support for App Engine, I was both ecstatic and intrigued about what might be possible. To get something running in just a few minutes, I decided to use our Twilio PHP helper.

When experimenting with a new Platform as a Service, there are nuances of which you should be aware like dealing with a virtualized file system and needing a separate service for email. However, the remaining nuances are usually pretty minimal and required only the “tweaking of my module” rather than a heavy “rebuilding of my app”.

Knowing that up front, let’s dig in.

Set up the PHP on App Engine environment

First, check out and follow Google’s Getting started with PHP on Google App Engine to set up your local environment. Their instructions will cover setting up the SDK, connecting to your account, and some details on debugging. It should set up your environment under http://localhost:8080/ which serves as the root of your application.

Upgrade your Twilio Helper library

Next, with respect to Twilio’s PHP Helper library, we’ve taken care of the nuances for you, the most important one of which involved falling back to PHP’s Streams when cUrl isn’t available. In your case, simply upgrade the library to v3.11+ or install it for the first time. You can use the library to send text messages and make phone calls exactly as you would in any other PHP environment:

<?php
// Include the Twilio PHP library
require "Services/Twilio.php";
 
// Set your AccountSid and AuthToken from www.twilio.com/user/account
$AccountSid = "ACXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX";
$AuthToken = "YYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYY";
 
// Instantiate a new Twilio Rest Client
$client = new Services_Twilio($AccountSid, $AuthToken);
 
// Make an array of people we know, to send them a message:
$people = array(
    "+14158675309" => "Curious George",
    "+14158675310" => "Boots",
    "+14158675311" => "Virgil",
);
 
// Loop over all our friends:
foreach ($people as $number => $name) {
 
    $sms = $client->account->sms_messages->create(
        "YYY-YYY-YYYY",     // Change the 'From' number to a Twilio number you've purchased
        $number,            // the number we are sending to - Any phone number
        "Hey $name, Monkey Party at 6PM. Bring Bananas!"  // the sms body
    );
 
    echo "Sent message to $name";        // Display a confirmation message
}

If you’re interested in the the specific changes for our library, you can explore the relevant pull requests here and here. I also have an article called “Preparing your PHP for App Engine” in next month’s php|architect magazine.

URL Routing in PHP

Within App Engine, the entire routing system is powered by the app.yaml file. If you’re familiar with development using frameworks like Zend or Symfony, defining your routes will come naturally and may be marginally more difficult than copy/paste. If you’re only familiar with doing non-framework development in PHP, you’ll have to define a route for each and every PHP file the user accesses. In Google’s example “hello world” app, your app.yaml file should should begin with something like this:

application: examplehelloworld
version: 1
runtime: php
api_version: 1
threadsafe: true
 
handlers:
- url: .*
  script: main.php

The request handlers are the important part. If you’re using an MVC framework or OpenVBX, you’ll most likely have one primary element mapping your index.php file and a set of elements for static assets like JavaScript, CSS, and images.

Alternatively, if you’re not using a using a framework, app.yaml will map different URL patterns to individual PHP scripts, like this:

application: examplehelloworld
version: 1
runtime: php
api_version: 1
threadsafe: true
 
handlers:
- url: /images
  static_dir: images
 
- url: /send-sms
  script: send-sms.php
 
- url: /make-call
  script: make-call.php
 
- url: .*
  script: index.php

Within those individual PHP scripts, no changes are necessary, so our make-call.php is straightforward and identical to our quickstart:

<?php
 
require 'Services/Twilio.php';
include 'credentials.php';
 
$client = new Services_Twilio($AccountSid, $AuthToken);
 
$call = $client->account->calls->create(
    '1512-555-1212', // From this number
    '17035551212',   // Send to this number
    'http://twimlets.com/echo?Twiml=%3CResponse%3E%3CSay%3EHello%20Monkey!%3C%2FSay%3E%3C%2FResponse%3E&'
);
print $call->sid;

Also, the send-sms.php can be the same as our normal quickstart:

<?php
 
require 'Services/Twilio.php';
include 'credentials.php';
 
$client = new Services_Twilio($AccountSid, $AuthToken);
 
$call = $client->account->sms_messages->create(
    '1512-555-1212', // From this number
    '17035551212',   // Send to this number
    'Hello monkey!!'
);
print $call->sid;

Now you can access your scripts via http://localhost:8080/make-call.php and http://localhost:8080/send-sms.php respectively. If the file is one that doesn’t need to be accessible publicly, such as your credentials.php file, it shouldn’t have a listing.

Voila. You now have your first PHP application on Google App Engine!

I hope this is helpful as you test and deploy your PHP applications on Google App Engine. If you have any tips, feedback, suggestions, or just a good joke please let me know via email or Twitter: @CaseySoftware.


Keith Casey currently serves as a Developer Evangelist for Twilio to get good tools to good developers so they can change how businesses communicate. In his spare time, he is a core contributor to web2project, works to build and support the Austin technology community, blogs occasionally at CaseySoftware.com and is completely fascinated by monkeys.

Posted by Ashleigh Rentz, Editor Emerita

Get started with App Engine for PHP: scalable, secure and reliable

By Andrew Jessup, Product Manager

Cross-posted from the Google Cloud Platform Blog

At Google I/O, we announced PHP as the latest supported runtime for Google App Engine in Limited Preview. PHP is one of the world's most popular programming languages, used by developers to power everything from simple web forms to complex enterprise applications.

Now PHP developers can take advantage of the scale, reliability and security features of App Engine. In addition, PHP runs well with other parts of Google Cloud Platform. Let's look at how this works.

Connecting to Google Cloud SQL from App Engine for PHP

Many PHP developers start with MySQL when choosing a database to store critical information, and a wide variety of products and frameworks such as WordPress make extensive use of MySQL’s rich feature set. Google Cloud SQL provides a reliable, managed database service that is MySQL 5.5 compatible and works well with App Engine.

To set up a Cloud SQL database, sign into Google Cloud Console - create a new project, choose Cloud SQL and create a new instance.

After you create the instance, it's automatically associated with your App Engine app.

You will notice Cloud SQL instances don’t need an IP address. Instead they can be accessed via a compound identifier made up of their project name and instance name, such as hello-php-gae:my-cloudsql-instance.

From within PHP, you can access Cloud SQL directly using the standard PHP MySQL libraries - mysql, mysqli or PDO_MySQL. Just specify your Cloud SQL database with its identifier, such as:

<?php

$db = new PDO(
  'mysql:unix_socket=/cloudsql/hello-php-gae:my-cloudsql-instance;dbname=demo_db;charset=utf8',
  'demo_user',
  'demo_password'
);

foreach($db->query('SELECT * FROM users') as $row) {
  echo $row['username'].' '.$row['first_name']; //etc...
}

Methods such as query() work just as you’d expect with any MySQL database. This example uses the popular PDO library, although other libraries such as mysql and mysqli work just as well.

Storing files with PHP and Google Cloud Storage

Reading and writing files is a common task in many PHP projects, whether you are reading stored application state, or generating formatted output (e.g., writing PDF files). The challenge is to find a storage system that is as scalable and secure as Google App Engine itself. Fortunately, we have exactly this in Google Cloud Storage (GCS).

The first step in setting up Google Cloud Storage is to create a bucket:

With the PHP runtime, we’ve implemented native support for GCS. In particular, we’ve made it possible for PHP’s native filesystem functions to read and write to a GCS bucket.

This code writes all prime numbers less than 2000 into a file on GCS:

<?php

$handle = fopen('gs://hello-php-gae-files/prime_numbers.txt','w');

fwrite($handle, "2");
for($i = 3; $i <= 2000; $i = $i + 2) {
  $j = 2;
  while($i % $j != 0) {
    if($j > sqrt($i)) {
      fwrite($handle, ", ".$i);
      break;
    }
    $j++;
  }
}

fclose($handle);

The same fopen() and fwrite() commands are used just as if you were writing to a local file. The difference is we’ve specified a Google Cloud Storage URL instead of a local filepath.

And this code reads the same file back into memory and pulls out the 100th prime number, using file_get_contents():

<?php

$primes = explode(",",
  file_get_contents('gs://hello-php-gae-files/prime_numbers.txt')
);

if(isset($primes[100]))
  echo "The 100th prime number is ".$primes[100];

And more features supported in PHP

Many of our most popular App Engine APIs are now supported in PHP, including our zero-configuration Memcache, Task Queues for asynchronous processing, Users API, Mail API and more. The standard features you’d expect from App Engine, including SSL support, Page Speed Service, versioning and traffic splitting are all available as well.

Open today in Limited Preview

Today we’re making App Engine for PHP available in Limited Preview. Read more about the runtime in our online documentation, download an early developer SDK, and sign up to deploy applications at https://cloud.google.com/appengine/php.


Andrew Jessup is a Product Manager at Google, working on languages and runtimes for Google App Engine.

Posted by Scott Knaster, Editor

Ushering in the next generation of computing at Google I/O

By Urs Hölzle, Senior Vice President, Technical Infrastructure, and Google Fellow

Cross-posted from the Google Cloud Platform Blog

Watch the video of the Cloud track kickoff.

Over the last fourteen years we have been developing some of the best infrastructure in the world to power Google’s global-scale services. With Google Cloud Platform, our goal is to open that infrastructure and make it available to any business or developer anywhere. Today, we are introducing improvements to the platform and making Google Compute Engine available for anyone to use.

Google Compute Engine - now available for everyone

Google Compute Engine provides a fast, consistently high-performance environment for running virtual machines. Later today, you’ll be able to go online to cloud.google.com and start using Compute Engine.

In addition, we’re introducing new Compute Engine features:

• Sub-hour billing charges for instances in one-minute increments with a ten-minute minimum, so you don’t pay for compute minutes that you don’t use
• Shared-core instances provide smaller instance shapes for low-intensity workloads
• Advanced Routing features help you create gateways and VPN servers, and enable you to build applications that span your local network and Google’s cloud
• Large persistent disks support up to 10 terabytes per volume, which translates to 10X the industry standard

We’ve also completed ISO 27001:2005 international security certification for Compute Engine, Google App Engine, and Google Cloud Storage.

Google App Engine adds the PHP runtime

App Engine 1.8.0 is now available and includes a Limited Preview of the PHP runtime - your top requested feature. We’re bringing one of the most popular web programming languages to App Engine so that you can run open source apps like WordPress. It also offers deep integration with other parts of Cloud Platform including Google Cloud SQL and Cloud Storage.

We’ve also heard that we need to make building modularized applications on App Engine easier. We are introducing the ability to partition apps into components with separate scaling, deployments, versioning and performance settings.

Introducing Google Cloud Datastore

Google Cloud Datastore is a fully managed and schemaless solution for storing non-relational data. Based on the popular App Engine High Replication Datastore, Cloud Datastore is a standalone service that features automatic scalability and high availability while still providing powerful capabilities such as ACID transactions, SQL-like queries, indexes and more.

Over the last year we have continued our focus on feature enhancement and developer experience across App Engine, Compute Engine, Google BigQuery, Cloud Storage and Cloud SQL. We also introduced Google Cloud Endpoints and Google Cloud Console.

With these improvements, we have seen increased usage with over 3 million applications and over 300,000 unique developers using Cloud Platform in a given month. Our developers inspire us everyday, and we can’t wait to see what you build next.


Urs Hölzle is Senior Vice President of Technical Infrastructure and Google Fellow. As one of Google's first ten employees and its first VP of Engineering, he has shaped much of Google's development processes and infrastructure.

Posted by Scott Knaster, Editor

Education Awards on Google App Engine

By Andrea Held, Google University Relations

Cross-posted from the Google Research Blog

Last year we invited proposals for innovative projects built on Google’s infrastructure. Today we are pleased to announce the 11 recipients of a Google App Engine Education Award. Professors and their students are using the award in cloud computing courses to study databases, distributed systems, web mashups and to build educational applications. Each selected project received $1000 in Google App Engine credits.

Awarding computational resources to classroom projects is always gratifying. It is impressive to see the creative ideas students and educators bring to these programs.
Below is a brief introduction to each project. Congratulations to the recipients!

John David N. Dionisio, Loyola Marymount University
Project description: The objective of this undergraduate database systems course is for students to implement one database application in two technology stacks, a traditional relational database and on Google App Engine. Students are asked to study both models and provide concrete comparison points.

Xiaohui (Helen) Gu, North Carolina State University
Project description: Advanced Distributed Systems Class
The goal of the project is to allow the students to learn distributed system concepts by developing real distributed system management systems and testing them on real world cloud computing infrastructures such as Google App Engine.

Shriram Krishnamurthi, Brown University
Project description: WeScheme is a programming environment that runs in the Web browser and supports interactive development. WeScheme uses App Engine to handle user accounts, serverside compilation, and file management.

Feifei Li, University of Utah
Project description: A graduate-level course that will be offered in Fall 2013 on the design and implementation of large data management system kernels. The objective is to integrate features from a relational database engine with some of the new features from NoSQL systems to enable efficient and scalable data management over a cluster of commodity machines.

Mark Liffiton, Illinois Wesleyan University
Project description: TeacherTap is a free, simple classroom-response system built on Google App Engine. It lets students give instant, anonymous feedback to teachers about a lecture or discussion from any computer or mobile device with a web browser, facilitating more adaptive class sessions.

Eni Mustafaraj, Wellesley College
Project description: Topics in Computer Science: Web Mashups. A CS2 course that combines Google App Engine and MIT App Inventor. Students will learn to build apps with App Inventor to collect data about their life on campus. They will use Google App Engine to build web services and apps to host the data and remix it to create web mashups. Offered in the 2013 Spring semester.

Manish Parashar, Rutgers University
Project description: Cloud Computing for Scientific Applications -- Autonomic Cloud Computing teaches students how a hybrid HPC/Grid + Cloud cyber infrastructure can be effectively used to support real-world science and engineering applications. The goal of our efforts is to explore application formulations, Cloud and hybrid HPC/Grid + Cloud infrastructure usage modes that are meaningful for various classes of science and engineering application workflows.

Orit Shaer, Wellesley College
Project description: GreenTouch
GreenTouch is a collaborative environment that enables novice users to engage in authentic scientific inquiry. It consists of a mobile user interface for capturing data in the field, a web application for data curation in the cloud, and a tabletop user interface for exploratory analysis of heterogeneous data.

Elliot Soloway, University of Michigan
Project description: WeLearn Mobile Platform: Making Mobile Devices Effective Tools for K-12. The platform makes mobile devices (Android, iOS, WP8) effective, essential tools for all-the-time, everywhere learning. WeLearn’s suite of productivity and communication apps enable learners to work collaboratively; WeLearn’s portal, hosted on Google App Engine, enables teachers to send assignments, review, and grade student artifacts. WeLearn is available to educators at no charge.

Jonathan White, Harding University
Project description: Teaching Cloud Computing in an Introduction to Engineering class for freshmen. We explore how well-designed systems are built to withstand unpredictable stresses, whether that system is a building, a piece of software or even the human body. The grant from Google is allowing us to add an overview of cloud computing as a platform that is robust under diverse loads.

Dr. Jiaofei Zhong, University of Central Missouri
Project description: By building an online Course Management System, students will be able to work on their team projects in the cloud. The system allows instructors and students to manage the course materials, including course syllabus, slides, assignments and tests in the cloud; the tool can be shared with educational institutions worldwide.


Andrea Held is a Program Manager on the University Relations team at Google. She grew up in Germany and has lived in California for almost 30 years.

Posted by Scott Knaster, Editor

Using Tailbone to talk to App Engine with JavaScript

By Doug Fritz, Creative Lab

Today we’re sharing a small open source project called Tailbone that lets developers read and write to the Google App Engine Datastore using JavaScript. We’re hoping that it makes App Engine a bit more accessible to developers who aren’t familiar with Python, Java or Go, or prefer not to use them.

I share an office with three creative programmers who work almost entirely in HTML5 and JavaScript. An important part of our work is writing server-side code for new projects that read or write data to to the App Engine Datastore or use Google accounts to store authenticated user-specific information. To make that process easier for my JavaScript-fluent colleagues, I created Tailbone to act as a RESTful API for an app’s Datastore.

To get started, you still have to install App Engine’s SDK and Python, but after that you’re all set. We’ve written a detailed tutorial that guides you through the installation and an example app for creating an authenticated profile page with an editable name and photo.

It’s my hope that Tailbone makes App Engine a little bit less intimidating for people who don’t have much experience with server-side coding. I know there are a few in my office. If there are any others out there, this is for you.


Doug Fritz is a programmer with the Creative Lab’s Data Arts Team. He thinks large amounts of data taste slightly purple and strongly wishes the government used bugzilla.

Posted by Scott Knaster, Editor

Improve your App Engine skills with Google Developers Academy

By Wesley Chun, Developer Relations Team

Cross-posted with the Google App Engine Blog

Are you developing on App Engine today or interested in learning how to use it? If you've gone through all the great App Engine docs and Getting Started tutorials (Python, Java, or Go) but want to take your App Engine skills a step further, then Google Developers Academy (GDA) is the place to go! We launched GDA this past summer at Google I/O 2012, with content for beginners as well as seasoned developers. What can you find on App Engine in GDA today?

If you’re interested in getting more background on what cloud computing is and where App Engine fits into that ecosystem, then this intro class (Introduction to Google App Engine) is for you. Once you’re done with this class, you’ll be ready to tackle the Getting Started tutorial, and after that, move on to the App Engine 101 in Python class.

While some of the material found in App Engine 101 is similar to what's in the Getting Started tutorial, the 101 class targets developers who skipped the tutorial or completed it at some point in the past but don't want to repeat the exact same thing. The main differences include the following changes to the tutorial's content:

• Use of the Python NDB API
• Jinja2 templates
• Discussion of data consistency and datastore indexes

You can use the relational MySQL-compatible Google Cloud SQL service as an alternative to App Engine's native non-relational datastore. Some applications do require a relational database, especially if you’re porting an existing app that relies on one. In this case, you want to learn about Cloud SQL and how to use it with App Engine. That’s why we have the Using Python App Engine with Google Cloud SQL class.

Of course, Google is best known for search. With App Engine's powerful Search API, you can index not only plain text, but also HTML, atoms, numbers, dates, and locations (lat/long). Getting Started with the Python Search API is a two-part class that will indeed get you started: in the first part of the class, you’ll create an application using a variety of data and learn how to index such data (using "documents"). In Part 2, you’ll learn how to execute queries as well as how to update your indexes when you modify your data.

If variety is what you're after, then look no further than the newest class in GDA: Getting Started with Go, App Engine and Google+ API. You will not only learn how to create an App Engine app using the Go programming language, but also learn how to connect to the Google+ API with the Google APIs Client Library for Go.

These are just a few examples of the types of classes you'll find in GDA. We also have content that features many other Google technologies, including Android, Chrome, YouTube, Maps, Drive, and Wallet. We invite you to swing by for a visit soon.


+Wesley Chun (@wescpy) is author of the bestselling Core Python books and a Developer Advocate at Google, specializing in cloud computing and academia. He loves traveling worldwide to meet Google users everywhere, whether at a developers conference, user group meeting, or on a university campus!

Posted by Scott Knaster, Editor

Google Cloud Platform introduces new support packages

By Brett McCully, Google Cloud Platform Team

(Cross-posted with the App Engine and Enterprise Blogs)

Support is as important as product features when choosing a platform for your applications. And let’s face it, sometimes we all need a bit of help. No matter which Google Cloud Platform services you are using — App Engine, Compute Engine, Cloud Storage, Cloud SQL, BigQuery, etc. — or what time of day, you should be able to get the answers you need. While you can go to Stack Overflow or Google Groups, we realize some of you may need 24x7 coverage, phone support or direct access to a Technical Account Manager team.

To meet your support requirements, we’re introducing a comprehensive collection of support packages for services on Google Cloud Platform, so you can decide what level best fits your needs:

• Bronze: All customers get access to online documentation, community forums, and billing support. (Free)
• Silver: In addition to Bronze, you can email our support team for questions related to product functionality, best practices, and service errors. ($150/month)
• Gold: In addition to Silver, you'll receive 24x7 phone support and consultation on application development, best practices or architecture for your specific use case. (Starts at $400/month)
• Platinum: The most comprehensive and personalized support. In addition to Gold, you’ll get direct access to a Technical Account Manager team. (Contact Sales for more information)

Sign up or click here to find out more information about the new Google Cloud Platform support options.

Brett McCully is the Manager of the Google Cloud Platform Support team and is currently based in Seattle.

Posted by Ashleigh Rentz, Editor Emerita

Java 7 Runtime and Cloud Endpoints Preview

By Brad Abrams, Google Cloud Platform Team
(Cross-posted with the Google App Engine Blog)
Today we are announcing two new preview features: App Engine Java 7 runtime support and Google Cloud Endpoints.   Preview features are ‘experimental’ features on a path towards general availability.

Java 7 Runtime Support for App Engine

The App Engine Java 7 runtime allows App Engine developers to keep pace with innovations in the Java language and runtime. It is important that you begin testing your applications with the new Java 7 runtime before the complete rollout in a few months.
Some of the language features you now have access to include:
invokedynamic support, which allows developers, tools providers, and language implementations to take advantage of a new bytecode, invokedynamic, to handle method invocations efficiently even when there is no static type information. For example:

  public static void invokeExample() {
    String s;
    MethodType mt; 
    MethodHandle mh;

    MethodHandles.Lookup lookup = MethodHandles.lookup();
    MethodType mt = MethodType.methodType(String.class, char.class,
                                          char.class);
    MethodHandle mh = lookup.findVirtual(String.class, "replace", mt);
    s = (String) mh.invokeExact("App Engine Java 6 runtime",'6','7');
    System.out.println(s);
  }

Try-with-resources, which helps avoid memory leaks and related bugs by automatically closing resources that are used in a try-catch statement.

  public static void viewTable(Connection con, String query) throws SQLException {

   try (
Statement stmt = con.createStatement();
ResultSet rs = stmt.executeQuery(query)
) {
      while (rs.next()) {
      // process results
      //
      }
   } catch (SQLException e) {
      // con resource is auto-closed, no need to do anything here!
      //
   }
}

Flexible Type Creation when using generics, enabling you to create parameterized types more succinctly. For example, you can write:

  Map<String, List<String>> myMap = new HashMap<>();

instead of:

Map<String, List<String>> myMap = new HashMap<String, List<String>>();

In addition to the language features listed above, the App Engine Java 7 runtime also includes:

• Use of String class in Switch statements.
• Expression of binary literals using simple prefixes 0b or 0B.
• Single catch blocks that can handle multiple exceptions.
• Simplified varargs method invocation.

Want to get started now? Download the latest App Engine Java SDK and then check out the docs here.

Cloud Endpoints Preview

Have you ever wanted a simple way to get a cloud backend for that Android or iPhone app you are working on?  Wish it was easier to expose RESTful APIs from your web application?  Google Cloud Endpoints simplifies building highly scalable and flexible backends for your web and mobile applications. Use Cloud Endpoints to store application data in the cloud that spans both devices and users. You can now easily expose your own authenticated, reliable, REST-based APIs hosted from an App Engine application and consume them in your Android, iOS, or web clients. Deeply integrated authentication support allows your users to have a transparent yet secure experience accessing your services. You have access to strongly typed client libraries for your custom service optimized for Android and iOS.

To use Cloud Endpoints, you simply write a Java or Python class with methods you want to expose to web or mobile clients. You then annotate the methods with attributes that control exactly how they are represented in REST interfaces on the wire. Finally, use Cloud Endpoints to generate your strongly-typed client libraries for Android, iOS and a lightweight JavaScript library.
For example, you can create a simple class to list some important data:

  public class SuperHeroes {

    public List listSuperHeroes() {
  
  List list = new ArrayList();
  list.add(new SuperHero ("Champion of the Obvious", "Brad Abrams"));
  list.add(new SuperHero ("Mr. Justice", "Chris Ramsdale"));

  return list;

 }
}

Then, expose it over a standard REST interface with a simple attribute and a versioning pattern.

@Api(name = "superheroes", version = "v1")
public class SuperHeroesV1 {
 ...
}

Now you have a simple REST interface.

$ curl http://localhost:8888/_ah/api/superheroes/v1/superheroes
{
 "items": [
  {
   "knownAs" : "Champion of the Obvious",
   "realName" : "Brad Abrams"
  },
  {
   "knownAs" : "Mr. Justice",
   "realName" : "Chris Ramsdale"
  }

And you can make strongly typed calls from your Android clients:

Real result = superheroes.list().execute();

Or Objective-C iOS client:

GTLQuerySuperHeroesV1 *query = [GTLQuerySuperHeroesV1 queryForSuperHeroesList];
[service executeQuery:query completionHandler:^(GTLServiceTicket *ticket, 
GTLSuperHeroes *object, NSError *error) {
  NSArray *items = [object items];
}];

Or the web client in JavaScript:

// ...
var ROOT = 'https://' + window.location.host + '/_ah/api';
gapi.client.load('superheroes', 'v1',
                 loadSuperheroesCallback, ROOT);

// Get the list of superheroes
gapi.client.superheroes.superheroes.list().execute(function(resp) {
  displaySuperheroesList(resp);
});

Read the documentation for Java or Python to discover how you can build a simple tic-tac-toe game using Cloud Endpoints.
To get started with Cloud Endpoints, download the App Engine 1.7.5 SDK and the latest Google Plugin for Eclipse. Be sure to look at the docs and follow along in the discussion forums on Stack Overflow.
For more on using Cloud Endpoints with Python, check out +Danny Hermes and +Dan Holevoet on Google Developers Live.


Brad Abrams is a Product Manager on the Google Cloud Platform where he looks after the developer experience. Brad is currently learning to ride the unicycle, so far with no broken bones!
Posted by Ashleigh Rentz, Editor Emerita

Research Projects on Google App Engine

By Andrea Held, Program Manager, Google University Relations

Last spring Google University Relations announced an open call for proposals for Google App Engine Research Awards. We invited academic researchers to use Google App Engine for research experiments and analysis, encouraging them to take advantage of the platform’s ability to manage heavy data loads and run large-scale applications. Submissions included proposals in various subject areas such as mathematics, computer vision, bioinformatics, climate and computer science. We selected seven projects and have awarded each $60,000 in Google App Engine credits recognizing their innovation and vision.

Today we would like to share a brief introduction of the winning projects and their Principal Investigators:

• K. Mani Chandy, Simon Ramo Professor and Professor of Computer Science, California Institute of Technology
  Cloud-based Event Detection for Sense and Response: Develop a low-cost alternative to traditional seismic networks. The image below is taken from the Community Seismic Network map showing active clients and events in real time.



A dense network of seismic stations enables the Community Seismic Network to perform a finer-grained analysis of seismic events than possible with existing seismic networks.

• Lawrence Chung, Associate Professor, The University of Texas at Dallas
  Google App Engine: Software Benchmark and Google App Engine Simulation Forecaster: Develop a tool to estimate software performance and cost on Google App Engine.
• Julian Gough, Professor, University of Bristol, UK
  Personalised DNA Analysis: Develop a service that provides personal DNA analysis.
• Ramesh Raskar, PhD, MIT Media Lab; Dr. Erick Baptista Passos, IFPI (Federal Institute of Technology, Brazil)
  Vision Blocks: develop a tool that delivers computer vision to people everywhere. The image below shows a current prototype implementation of Vision Blocks.



Many algorithms are already included, and you'll be able create your own blocks as well.

• Norman Sadeh, Professor, Director of Mobile Commerce Lab, School of
  Computer Science, Carnegie Mellon University
  Mapping the Dynamics of a City & Nudging Twitter Users: uncovering local collective knowledge about the a city using social media.
• William Stein, Professor of Mathematics, University of Washington
  Sage: Creating a Viable Free Open Source Alternative to Magma, Maple, Matlab, and Mathematica.
• Enrique Vivoni, Associate Professor, Hydrologic Science, Engineering & Sustainability, Arizona State University
  Cloud Computing-Based Visualization and Access of Global Climate Data Sets: provide scientific data on global climate trends.

Congratulations to the award winners! We are excited about the proposals’ creativity and innovation and look forward to learning about their discoveries. To read more about specific projects, go here.


Andrea Held is a Program Manager on the University Relations team at Google. She grew up in Germany and has lived in California for almost 30 years.

Posted by Scott Knaster, Editor

Find sample code and more for Google Cloud Platform, now on GitHub

By Julia Ferraioli, Developer Advocate, Google Compute Engine

Cross-posted from the Google Open Source Blog

Today, we’re announcing that you can now find Google Cloud Platform on GitHub! The GitHub organization for the Google Cloud Platform is your destination for samples and tools relating to App Engine, BigQuery, Compute Engine, Cloud SQL, and Cloud Storage. Most Google Cloud Platform existing open source tools will be migrated to the organization over time. You can quickly get your app running by forking any of our repositories and diving into the code.

Currently, the GitHub organization for the Google Cloud Platform has 36 public repositories, some of which are currently undergoing their initial code reviews, which you can follow on the repo. The Google Cloud Platform Developer Relations Team will be using GitHub to maintain our starter projects, which show how to get started with our APIs using different stacks. We will continue to add repositories that illustrate solutions, such as the classic guest book app on Google App Engine. For good measure, you will also see some tools that will make your life easier, such as an OAuth 2.0 helper.

From getting started with Python on Google Cloud Storage to monitoring your Google Compute Engine instances with App Engine, our GitHub organization is home to it all.

Trick of the trade: to find samples relating to a specific platform, try filtering on the name in the “Find a Repository” text field.

We set up this organization not only to give you an easy way to find and follow our samples, but also to give you a way to get involved and start hacking alongside us. We’ll be monitoring our repositories for any reported issues as well as for pull requests. If you’re interested in seeing what a code review looks like for Google’s open source code, you can follow along with the discussion happening right on the commits.

Let us know about your suggestions for samples. We look forward to seeing what you create!


Julia Ferraioli is a Developer Advocate for Google Compute Engine, based in Seattle. She helps developers harness the power of Google's infrastructure to tackle their computationally intensive processes and jobs. She comes from an industrial background in software engineering, and an academic background in machine learning and assistive technology.

Posted by Scott Knaster, Editor

Calling student coders: Hardcode, the secure coding contest for App Engine

By Parisa Tabriz, Security Team

Cross-posted from the Google Online Security Blog

Protecting user security and privacy is a huge responsibility, and software security is a big part of it. Learning about new ways to “break” applications is important, but learning preventative skills to use when “building” software, like secure design and coding practices, is just as critical. To help promote secure development habits, Google is once again partnering with the organizers of SyScan to host Hardcode, a secure coding contest on the Google App Engine platform.

Participation will be open to teams of up to 5 full-time students (undergraduate or high school, additional restrictions may apply). Contestants will be asked to develop open source applications that meet a set of functional and security requirements. The contest will consist of two rounds: a qualifying round over the Internet, with broad participation from any team of students, and a final round, to be held during SyScan on April 23-25 in Singapore.

During the qualifying round, teams will be tasked with building an application and describing its security design. A panel of judges will assess all submitted applications and select the top five to compete in the final round.

At SyScan, the five finalist teams will be asked to develop a set of additional features and fix any security flaws identified in their qualifying submission. After two more days of hacking, a panel of judges will rank the projects and select a grand prize winning team that will receive $20,000 Singapore dollars. The 2nd-5th place finalist teams will receive $15,000, $10,000, $5,000, and $5,000 Singapore dollars, respectively.

Hardcode begins on Friday, January 18th. Full contest details will be be announced via our mailing list, so subscribe there for more information.


Written by Parisa Tabriz, Security Team.

Posted by Scott Knaster, Editor

App Engine 1.7.4 released

By the Google App Engine Team

Cross-posted from the Google App Engine Blog

The Google App Engine team has been busy putting together our final release of 2012. This release includes a number of features graduating from Experimental status as well as the usual batch of bug fixes and improvements. We’ll be taking a short break from our monthly release cycle over the holidays, but we’ll be back to our normal schedule starting in February.  

Expanded EU Support
We’re happy to announce that we are expanding European Union datacenter support, based on positive feedback from early users.  You can sign up here.  Please note, deployment is currently limited to billing-enabled applications.
We understand that data locality and latency are important to developers and are committed to further expanding this support in the coming months.

Java
We've made a new Maven plugin available, added the source to the SDK for easier IDE debugging, and made significant performance improvements to the JSP compilation process.  Please see our Java release notes for more information.

Python
We’ve made a big push to bring a number of new features to GA, upgraded the interpreter to version 2.7.3, and added several new experimental features.  For more details, please see our Python release notes.

New and Upgraded Features
We also have a handful of notable features in this release:

• Task Queue statistics (General Availability): You can see statistics such as the current number of tasks in a queue, the number of tasks executed in the last hour, and more.
• Traffic splitting (General Availability): You can split requests amongst different versions of your app.
• LogsReader and Logs API (General Availability): You can now fetch requests based on a list of request IDs.
• Expanded Datastore query support (Experimental): We’ve added ‘DISTINCT’ support to Datastore queries.

Full release notes and Google Cloud Platform newsletter
The complete list of features and a list of bug fixes for 1.7.4 can be found in our release notes. For App Engine coding questions and answers check us out on Stack Overflow, and for general discussion and feedback, find us on our Google Group.

Finally, to stay up to date with Google Cloud Platform, sign up for our newsletter where you’ll get the latest news, announcements and event information.


Written by the Google App Engine Team.

Posted by Scott Knaster, Editor

Google HackFair in South Korea

By Soonson Kwon, Developer Relations Program Manager

For developers and engineers, the best way to learn something is to get your hands dirty and try making something. That is why Google hosts many hackathons around the world. Last November 17 and 18, we had a bigger experiment at Gangnam (yes, this is the very Gangnam in Gangnam Style!) in Seoul, South Korea which expanded a 1-2 day hackathon into a much longer one which we called Google HackFair.



The idea was to give developers enough time (2 months) to develop something bigger and provide a nice chance to showcase their projects. 153 developers submitted 92 projects, and 40 projects were chosen from among them and displayed. Developers used many different technologies, including Android, Chrome, App Engine, and HTML5, and they completed creative and interesting projects: a remote controlled car guided by Android, a serial terminal for Chrome, a braille printer using Go, and many more!

Besides the exhibition, we also prepared a mini-conference and GDG (Google Developers Group) booth where Googlers and community developers gave 27 sessions in total.

More than 1000 people attended and enjoyed the Google HackFair. Although the event is finished, developers continue updating and polishing their projects. It was a great time indeed.

If you are interested in details for the projects including full demos or source code, please check here.


Soonson Kwon is Developer Relations Program Manager and Country Lead for South Korea. His mission is to help Korean developers make better use of Google’s developer products. He is also passionate about Open Source.

Posted by Scott Knaster, Editor

Google Cloud Storage - more value for performance

By Dave Barth, Product Manager

Cross-posted with the Official Google Enterprise Blog

Earlier this week, we announced a collection of improvements across Google Cloud Platform including 36 new Compute Engine instances, Durable Reduced Availability (DRA) storage, Object Versioning, and European datacenter support. We also announced that we are reducing the price of standard Google Cloud Storage by over 20%.

We are committed to delivering the best value in the marketplace to businesses and developers looking to operate in the cloud. That’s why today we are reducing the price of Google Cloud Storage by an additional 10%, resulting in a total price reduction of over 30%. This price reduction applies to all Cloud Storage regions and the new DRA Storage.

Find out more about the new Cloud Storage pricing and sign up now to get started.

Dave Barth is a Product Manager on the Google Cloud Storage team, based in Seattle. He is idealistic about the capacity of technology to change the world.

Posted by Raj Sarkar

Google Cloud Platform: new features, lower prices, extending European datacenters

By Jessie Jiang, Product Management Director

(Cross-posted on the Official Google Enterprise Blog.)

We're constantly making updates to our Google Cloud Platform products—Google App Engine, Cloud Storage, Big Query, Compute Engine and others—based on user feedback and to improve the overall experience. For example, two weeks ago we introduced a major update to Google Cloud SQL providing faster performance, larger databases (100GB), an EU zone, and a no-cost trial. But, we know there is more to do. Today, we’re continuing to improve the platform with new storage and compute capabilities, significantly lower prices, and more European Datacenter support.

Lower storage prices and new Durable Reduced Availability (DRA) Storage

Updated 3:23 PM to provide more details about DRA. To give you more flexibility in your storage options and prices, we’re reducing the price of standard Google Cloud Storage by over 20% and introducing a limited preview of Durable Reduced Availability (DRA) storage. DRA storage lowers prices by reducing some data availability, while maintaining the same latency performance and durability as standard Google Cloud Storage. This makes it a great option for batch compute jobs that can easily be rescheduled or for data backup where quick access to your data is important. DRA achieves cost savings by keeping fewer redundant replicas of data. Unlike other reduced redundancy cloud storage offerings, DRA is implemented in a manner that maintains data durability so you don't have to worry about losing your data in the cloud.

And, to automatically keep a history of old versions of your data, we’re introducing Object Versioning. You can also use it to help protect against deleting or overwriting your data by mistake or due to an application error.

More European Datacenter support
We are continuing to roll out our European Datacenter support. Now, customers using Google App Engine, Google Cloud Storage, Google Cloud SQL and (soon) Google Compute Engine can deploy their applications, data and virtual machines to European Datacenters. This helps bring your solutions even closer to your customers for faster performance and enables international redundancy.

36 New Compute Engine instance types and overall reduced prices
Earlier this year we introduced a Limited Preview of Google Compute Engine with four standard instance types. Today, we are announcing 36 additional instance types and are reducing the price of our original 4 standard instances by about 5% for those currently in our preview. In the coming weeks, the following will be available:

• High Memory Instance - High performance instances tailored for applications that demand large amounts of memory.
• High CPU Instance - Reduced cost option when applications don’t require as much memory.
• Diskless Configurations - Lower cost options for applications that do not require ephemeral disk and can exclusively utilize persistent disk.

We are also introducing Persistent Disk Snapshotting which makes it simple to instantly create a backup of your disk, move it around Google datacenters, and use the snapshot to start up a new VM.

We want to thank you, the community of developers and businesses who are pushing the platform into new areas and building innovative applications. We look forward to seeing where you take it next. Find out more about the new Cloud Storage pricing and Compute Engine instances. Sign up now and get started today.


Jessie Jiang is the Product Management Director, Google Cloud Platform. She is passionate about building the best platform for developers and businesses in the cloud.

Posted by Scott Knaster, Editor

Streak brings CRM to the inbox with Google Cloud Platform

By Aleem Mawani, Co-Founder of Streak

Cross-posted with the Google App Engine Blog

This guest post was written by Aleem Mawani, Co-Founder of Streak, a startup alum of Y Combinator, a Silicon Valley incubator. Streak is a CRM tool built into Gmail. In this post, Aleem shares his experience building and scaling their product using Google Cloud Platform.

Everyone relies on email to get work done – yet most people use separate applications from their email to help them with various business processes. Streak fixes this problem by letting you do sales, hiring, fundraising, bug tracking, product development, deal flow, project management and almost any other business process right inside Gmail. In this post, I want to illustrate how we have used Google Cloud Platform to build Streak quickly, scalably and with the ability to deeply analyze our data.

We use several Google technologies on the backend of Streak:

• App Engine to serve our app.

• App Engine Datastore to persist user data.

• Memcache to make operations fast.

• BigQuery to analyze our logs and power dashboards.

• App Engine Search API to let users sift through their data.

• Prediction API to machine learn over user data.

• Google Translate API to translate our app to over 40 languages.

Our core learning is that you should use the best tool for the job. No one technology will be able to solve all your data storage and access needs. Instead, for each type of functionality, you should use a different service. In our case, we aggressively mirror our data in all the services mentioned above. For example, although the source of truth for our user data is in the App Engine Datastore, we mirror that data in the App Engine Search API so that we can provide full text search, Gmail style, to our users. We also mirror that same data in BigQuery so that we can power internal dashboards.

System Architecture

App Engine - We use App Engine for Java primarily to serve our application to the browser and mobile clients in addition to serving our API. App Engine is the source of truth for all our data, so we aggressively cache using Memcache. We also use Objectify to simplify access to the Datastore, which I highly recommend.

Google Cloud Storage - We mirror all of our Datastore data as well as all our log data in Cloud Storage, which acts as a conduit to other Google cloud services. It lets us archive the data as well as push it to BigQuery and the Prediction API.

BigQuery - Pushing the data into BigQuery allows us to run non-realtime queries that can help generate useful business metrics and slice user data to better understand how our product is getting used. Not only can we run complex queries over our Datastore data but also over all of our log data. This is incredibly powerful for analyzing the request patterns to App Engine. We can answer questions like:

• Which requests cost us the most money?
• What is the average response time for every URL on our site over the last 3 days?

BigQuery helps us monitor error rates in our application. We process all of our log data with debug statements, as well as something called an “error type” for any request that fails. If it’s a known error, we'll log something sensible, and we log the exception type if we haven’t seen it before. This is beneficial because we built a dashboard that queries BigQuery for the most recent errors in the last hour grouped by error type. Whenever we do a release, we can monitor error rates in the application really easily.

A Streak dashboard powered by BigQuery showing current usage statistics

In order to move the data into Cloud Storage from the Datastore and LogService, we developed an open source library called Mache. It’s a drop-in library that can be configured to automatically push data into BigQuery via Cloud Storage. The data can come from the Datastore or from LogService and is very configurable - feel free to contribute and give us feedback on it!

Google Cloud Platform also makes our application better for our users. We take advantage of the App Engine Search API and again mirror our data there. Users can then query their Streak data using the familiar Gmail full text search syntax, for example, “before:yesterday name:Foo”. Since we also push our data to the Prediction API, we can help users throughout our app by making smart suggestions. In Streak, we train models based on which emails users have categorized into different projects. Then, when users get a new email, we can suggest the most likely box that the email belongs to.

One issue that arises is how to keep all these mirrored data sets in sync. It works differently for each service based on the architecture of the service. Here’s a simple breakdown:

Having these technologies easily available to us has been a huge help for Streak. It makes our products better and helps us understand our users. Streak’s user base grew 30% every week for 4 consecutive months after launch, and we couldn’t have scaled this easily without Google Cloud Platform. To read more details on why Cloud Platform makes sense for our business, check out our case study and our post on the Google Enterprise blog.


Aleem Mawani is the co-founder of Streak.com, a CRM tool built into Gmail. Previously, Aleem worked on Google Drive and various ads products at Google. He has a degree from the University of Waterloo in Software engineering and an MBA from Harvard University.

Posted by Scott Knaster, Editor