Securely storing environment variables in GAE with app.yaml

Securely storing environment variables in GAE with app.yaml

I need to store API keys and other sensitive information in app.yaml as environment variables for deployment on GAE. The issue with this is that if I push app.yaml to GitHub, this information becomes public (not good). I don't want to store the info in a datastore as it does not suit the project. Rather, I'd like to swap out the values from a file that is listed in .gitignore on each deployment of the app.

Here is my app.yaml file:

application: myapp  version: 3   runtime: python27  api_version: 1  threadsafe: true    libraries:  - name: webapp2    version: latest  - name: jinja2    version: latest    handlers:  - url: /static    static_dir: static    - url: /.*    script: main.application      login: required    secure: always  # auth_fail_action: unauthorized    env_variables:    CLIENT_ID: ${CLIENT_ID}    CLIENT_SECRET: ${CLIENT_SECRET}    ORG: ${ORG}    ACCESS_TOKEN: ${ACCESS_TOKEN}    SESSION_SECRET: ${SESSION_SECRET}  

Any ideas?

Answer by Gwyn Howell for Securely storing environment variables in GAE with app.yaml

Best way to do it, is store the keys in a client_secrets.json file, and exclude that from being uploaded to git by listing it in your .gitignore file. If you have different keys for different environments, you can use app_identity api to determine what the app id is, and load appropriately.

There is a fairly comprehensive example here -> https://developers.google.com/api-client-library/python/guide/aaa_client_secrets.

Here's some example code:

# declare your app ids as globals ...  APPID_LIVE = 'awesomeapp'  APPID_DEV = 'awesomeapp-dev'  APPID_PILOT = 'awesomeapp-pilot'    # create a dictionary mapping the app_ids to the filepaths ...  client_secrets_map = {APPID_LIVE:'client_secrets_live.json',                        APPID_DEV:'client_secrets_dev.json',                        APPID_PILOT:'client_secrets_pilot.json'}    # get the filename based on the current app_id ...  client_secrets_filename = client_secrets_map.get(      app_identity.get_application_id(),      APPID_DEV # fall back to dev      )    # use the filename to construct the flow ...  flow = flow_from_clientsecrets(filename=client_secrets_filename,                                 scope=scope,                                 redirect_uri=redirect_uri)    # or, you could load up the json file manually if you need more control ...  f = open(client_secrets_filename, 'r')  client_secrets = json.loads(f.read())  f.close()  

Answer by therewillbesnacks for Securely storing environment variables in GAE with app.yaml

It sounds like you can do a few approaches. We have a similar issue and do the following (adapted to your use-case):

• Create a file that stores any dynamic app.yaml values and place it on a secure server in your build environment. If you are really paranoid, you can asymmetrically encrypt the values. You can even keep this in a private repo if you need version control/dynamic pulling, or just use a shells script to copy it/pull it from the appropriate place.
• Pull from git during the deployment script
• After the git pull, modify the app.yaml by reading and writing it in pure python using a yaml library

The easiest way to do this is to use a continuous integration server such as Hudson, Bamboo, or Jenkins. Simply add some plug-in, script step, or workflow that does all the above items I mentioned. You can pass in environment variables that are configured in Bamboo itself for example.

In summary, just push in the values during your build process in an environment you only have access to. If you aren't already automating your builds, you should be.

Another option option is what you said, put it in the database. If your reason for not doing that is that things are too slow, simply push the values into memcache as a 2nd layer cache, and pin the values to the instances as a first-layer cache. If the values can change and you need to update the instances without rebooting them, just keep a hash you can check to know when they change or trigger it somehow when something you do changes the values. That should be it.

Answer by Zig Mandel for Securely storing environment variables in GAE with app.yaml

Note that unless you really want to use a public github, google now includes private git repositories in cloud projects which are way more practical as any dev can access and push-to-deploy all from a browser without needing access to other private networks or servers where the secret stuff is.

Answer by Bernd Verst for Securely storing environment variables in GAE with app.yaml

My approach is to store client secrets only within the App Engine app itself. The client secrets are neither in source control nor on any local computers. This has the benefit that any App Engine collaborator can deploy code changes without having to worry about the client secrets.

I store client secrets directly in Datastore and use Memcache for improved latency accessing the secrets. The Datastore entities only need to be created once and will persist across future deploys. of course the App Engine console can be used to update these entities at any time.

There are two options to perform the one-time entity creation:

• Use the App Engine Remote API interactive shell to create the entities.
• Create an Admin only handler that will initialize the entities with dummy values. Manually invoke this admin handler, then use the App Engine console to update the entities with the production client secrets.

Answer by Martin for Securely storing environment variables in GAE with app.yaml

If it's sensitive data, you should not store it in source code as it will be checked into source control. The wrong people (inside or outside your organization) may find it there. Also, your development environment probably uses different config values from your production environment. If these values are stored in code, you will have to run different code in development and production, which is messy and bad practice.

In my projects, I put config data in the datastore using this class:

from google.appengine.ext import ndb    class Settings(ndb.Model):    name = ndb.StringProperty()    value = ndb.StringProperty()      @staticmethod    def get(name):      NOT_SET_VALUE = "NOT SET"      retval = Settings.query(Settings.name == name).get()      if not retval:        retval = Settings()        retval.name = name        retval.value = NOT_SET_VALUE        retval.put()      if retval.value == NOT_SET_VALUE:        raise Exception(('Setting %s not found in the database. A placeholder ' +          'record has been created. Go to the Developers Console for your app ' +          'in App Engine, look up the Settings record with name=%s and enter ' +          'its value in that record\'s value field.') % (name, name))      return retval.value  

Your application would do this to get a value:

API_KEY = Settings.get('API_KEY')  

If there is a value for that key in the datastore, you will get it. If there isn't, a placeholder record will be created and an exception will be thrown. The exception will remind you to go to the Developers Console and update the placeholder record.

I find this takes the guessing out of setting config values. If you are unsure of what config values to set, just run the code and it will tell you!

The code above uses the ndb library which uses memcache and the datastore under the hood, so it's fast.

Answer by jla for Securely storing environment variables in GAE with app.yaml

You can use the -E command line option of appcfg.py to setup the environment variables when you deploy your app to GAE (appcfg.py update)

$ appcfg.py  ...  -E NAME:VALUE, --env_variable=NAME:VALUE                      Set an environment variable, potentially overriding an                      env_variable value from app.yaml file (flag may be                      repeated to set multiple variables).  ...  

Fatal error: Call to a member function getElementsByTagName() on a non-object in D:\XAMPP INSTALLASTION\xampp\htdocs\endunpratama9i\www-stackoverflow-info-proses.php on line 72