Project specific data storage

Related Tags: cloudconnect data loading

This article describes GoodData project-specific data storage, which can be used to stage and deliver your data files to your GoodData projects in the platform.

  • GoodData also provides storage for individual users with a valid GoodData platform account. For more information, see User-specific Storage.

Overview

GoodData Storage is cloud storage for staging your data before executing ETL processes on it.

NOTE: Project-specific storage is not permanent storage. Files are automatically deleted after 7 days.

How to use

  • You need to have your GoodData credentials (username and password).
  • You need to be an admin for the relevant GoodData project.
  • You need to have the Id of the relevant GoodData project.

Supported data file types

  • Valid row-based CSV
  • Well formed XML (adding a schema is helpful)

Access

You can access GoodData Storage through command-line utilities, WebDAV client software (i.e. Cyberduck), or Finder (on Mac OS X).

Required credentials

SERVER: secure-di.gooddata.com
USERNAME: Your GoodData username in the following format:
joe.user@gooddata.com
PASSWORD: Your GoodData password
PROJECT: Internal ID of your GoodData project, which should be provided by your implementation team.

NOTE: If you don’t have your GoodData account yet, get registered.

Accessing via API

You can also access the project uploads location via API. The following curl command returns the link to the project staging area:

curl -H 'Accept: application/json' -X GET https://secure.gooddata.com/gdc/projects/{project-id}/ | jq  '.project.links.uploads'

Instructions for Access

1. Cyberduck with WebDAV

Using Cyberduck client to connect via WebDAV:

WebDAV via Cyberduck
WebDAV via Cyberduck

2. OSX Finder

Steps:

  1. Open the Finder window.
  2. From the Finder menu, select Go > Connect to Server.
  3. Fill in your server address (https://secure-di.gooddata.com/project-uploads/PROJECT_ID/):
    OSX
    via OSX
  4. Click Connect. Use your GoodData email and password to authenticate.

NOTE: If you are accessing the storage via Browser. Don’t forget to type the “/” to the end of the URL. So the URL will be https://secure-di.gooddata.com/project-uploads/PROJECT_ID/

Uploading file using Ruby

require 'uri'
require 'net/http'

uri = URI.parse("https://secure-di.gooddata.com/project-uploads/vu43fz255ckul8pa4jofc81sr9boqub0/")
file = 'test.txt'
http = Net::HTTP.new(uri.host, uri.port)
http.use_ssl = true
request = Net::HTTP::Put.new("#{uri.request_uri}/test.txt")
request.basic_auth 'username@company.com', 'XXXX'
request.body_stream = File.open(file)
request["Content-Type"] = "multipart/form-data"
request.add_field('Content-Length', File.size(file))
response = http.request(request)

NOTE: When your file is zipped, we cannot unzip it until it is all uploaded. This ensures we don’t start transforming your files before you finish your uploading. You also save the bandwidth and time.