Gateway Readme for Linux

Table of Contents

  1. Overview
  2. System Requirements
  3. Set Up Instructions
    1. STEP 1: Create an Agent
      1. Starting the Agent
      2. Managing the Agent
    2. STEP 2: Add and Configure a Plugin 
      1. Option 1: Start from the Command Line
      2. Option 2: Start from the Socrata site
      3. Reconfigure a plugin
      4. Removing a plugin

Overview

Establishing a Socrata Gateway connection is a two-step process:

  1. Create an agent
  2. Add a plugin (or plugins) to that agent

Agents are small programs that establish a secure connection to Socrata and act as a manager for a plugin or set of plugins.

After the agent is downloaded and set up, we’ll help you connect to your individual data sources with plugins for specific data types.

You can add as many plugins as you like to a single agent, including multiple plugins of the same type.

System Requirements

The agent requires Java 8 or newer

  • Install Java from your system package manager
    • Debian/Ubuntu: sudo apt install openjdk-11-jre
    • Red Hat or CentOS: sudo yum install java-11-openjdk

Networking
The host must be allowed to make connections to Socrata on port 443

SSL
Special SSL and Self-Signed Certificates must be added to the Java trust store. For more information, please see: http://www.thesqlreport.com/?p=576

User Permissions
Sudo (root) or Administrator access is required to configure the service

Set Up Instructions

STEP 1: Create an Agent

Starting the Agent

  1. From your Socrata site click the “Provision Agent” button on either the:
    1. Admin/Gateway page ({Your_Domain}/admin/gateway) OR
    2. When you're choosing your data source while creating or editing a dataset
  2. Follow the agent naming and download process.
  3. Place the downloaded folder in a directory on the server or computer you are connecting to Socrata (if it’s not there already). Note or copy this folder path. You’ll need it when you set up a plugin later.
  4. Unzip the downloaded file.
  5. Run `linux/install` in that directory. It will ask a few questions about where and how to run the service, and then set it up to be run automatically by `systemd`.

     

 $ linux/install
systemd service name: example
User to run as: myuser
Directory to run in: /home/myuser/example-agent
Data directory: /home/myuser/.local/share/socrata-ingress-agent
Here is the unit file we will generate:

[Unit]
Description=The Socrata Ingress Agent
StartLimitIntervalSec=1
StartLimitBurst=5
[Service]
User=myuser
Group=myuser
WorkingDirectory=/home/myuser/example-agent
ExecStart=/usr/bin/java -jar "/home/myuser/example-agent/example.jar" --data-dir "/home/myuser/.local/share/socrata-ingress-agent"
Restart=on-failure
RestartSec=1
PrivateTmp=true
PrivateDevices=true
[Install]
WantedBy=multi-user.target

Is this correct? [Y/n] Y
created /home/myuser/Downloads/gateway_example/example.service
created /home/myuser/Downloads/gateway_example/example-control

Setting up the service. The commands being run will be displayed.
You may be asked for the root password, depending on your local
sudo configuration.

sudo mkdir -m0755 -p -- /home/myuser/example-agent
sudo cp -- /home/myuser/Downloads/gateway_example/socrata-ingress-agent.jar /home/myuser/example-agent/example.jar
sudo chmod 644 -- /home/myuser/example-agent/example.jar
sudo mkdir -m0700 -p -- /home/myuser/.local/share/socrata-ingress-agent
sudo chown -- myuser:myuser /home/myuser/.local/share/socrata-ingress-agent
sudo cp -- /home/myuser/Downloads/gateway_example/example.service /etc/systemd/system/example.service
sudo systemctl enable -- example.service
Created symlink /etc/systemd/system/multi-user.target.wants/example.service → /etc/systemd/system/example.service.
sudo systemctl start -- example.service

Setup complete. You may use the /home/myuser/Downloads/gateway_example/example-control
script to control, configure, or uninstall the service.

 

On your Socrata site you can check the agent status in the following places:

  • In the “Provision Agent” set up stepper (On the “Set up instructions” page there is an “Am I connected?” widget. Clicking “refresh” will check your connection.
  • On the Admin/Gateway page ({Your_Domain}/admin/gateway), it will appear in the table
  • When you're choosing your data source while creating or editing a dataset it will appear in the list as “Connected”

Managing the Agent

After installing the agent, you can use the generated control script
to manage it:

$ ./example-control
Usage: ./example-control stop|start|status|logs|run|uninstall

stop - stop the service
Important: All plugins associated with this agent will also stop and any schedules associated with datasets created under this agent will stop/break
start - start the service
status - display information about the service
logs - show the service's logs
-f, --follow output appended data as more is generated
run ARGUMENTS - run a command on the service jar
Use "./example-control run --help" for more information
uninstall - stop the service and delete all its data files

$ ./example-control start
$ ./example-control stop
$ ./example-control status


STEP 2: Add and Configure a Plugin

TIPS:

  • ”PLUGIN_TYPE” is a type from a list you’ll be presented upon set up.
  • “PLUGIN_NAME” is a name you select to identify this plugin on Socrata.
  • Configuration for the plugin type and plugin name are case sensitive. Please ensure you’re using the case presented at set up.
  • You can add as many plugins as you like to a single agent, including multiple plugins of the same type.

Option 1: Start from the Command Line

Open the command prompt on your computer or server and navigate to the directory where the downloaded agent jar file resides.

  1. To list available plugin types
    1. $ ./example-control run --list-available-plugins
  2. To install or reconfigure a plugin
    1. $ ./example-control run --configure-plugin PLUGIN_TYPE:PLUGIN_NAME
  3. When the plugin is run it will display a window on your server asking for plugin-specific configuration and credentials (if needed). This information will not live on Socrata.
  4. Fill out the configuration information and click “OK”. Return to Socrata and your plugin will then be ready to use!


Option 2: Start from the Socrata site

  1. Click "Add Plugin" either
    1. On the Admin/Gateway page ({Your_Domain}/admin/gateway)
    2. When you're choosing your data source while creating or editing a dataset
  2. Choose your plugin.
  3. Name your plugin.
    1. Naming tip: Who is responsible + Data source description + Data source type
    2. Example: Finance_Risk_CSV
    3. Non alphanumeric characters will be converted to underscores
  4. Open the command prompt on your computer or server and navigate to the directory where the downloaded agent jar file resides.
  5. Copy and paste the plugin command into the command prompt.
  6. Run the command by hitting enter or return. This will download the plugin from Socrata, verify it, and run it on your server.
    Tip: If you get an error, try changing directories.
  7. When the plugin is run it will display a window on your server asking for plugin-specific configuration and credentials (if needed). This information will not live on Socrata.
  8. Fill out the configuration information and click “OK”. Return to Socrata and your plugin will then be ready to use!


Reconfigure a plugin

  1. On the computer or server your plugin has been configured, in the command prompt enter:
    1. $ ./example-control run --configure-plugin PLUGIN_TYPE:PLUGIN_NAME
  2. When the plugin is run it will display a window on your server asking for plugin-specific configuration and credentials (if needed). This information will not live on Socrata.
  3. Fill out the configuration information and click “OK”. Return to Socrata and your reconfigured plugin will then be ready to use!

Removing a plugin

Important: All schedules associated with the dataset(s) created under this plugin will break when you remove the plugin.

  1. On the computer or server your plugin has been configured, in the command prompt enter:
    1. $ ./example-control run --delete-plugin PLUGIN_TYPE:PLUGIN_NAME
Was this article helpful?
0 out of 0 found this helpful
Have more questions? Submit a request

Comments

0 comments

Article is closed for comments.