Per-kernel overrides#
As mentioned in the overview of Process Proxy Configuration
capabilities, it’s possible to override or amend specific system-level configuration values on a per-kernel basis. These capabilities can be implemented with the kernel specification’s process-proxy config
stanza or via environment variables.
Per-kernel configuration overrides#
The following enumerates the set of per-kernel configuration overrides:
remote_hosts
: This process proxy configuration entry can be used to override--EnterpriseGatewayApp.remote_hosts
. Any values specified in the config dictionary override the globally defined values. These apply to allDistributedProcessProxy
kernels.yarn_endpoint
: This process proxy configuration entry can be used to override--EnterpriseGatewayApp.yarn_endpoint
. Any values specified in the config dictionary override the globally defined values. These apply to allYarnClusterProcessProxy
kernels. Note that you’ll likely be required to specify a differentHADOOP_CONF_DIR
setting in the kernel.json’senv
stanza in order of thespark-submit
command to target the appropriate YARN cluster.authorized_users
: This process proxy configuration entry can be used to override--EnterpriseGatewayApp.authorized_users
. Any values specified in the config dictionary override the globally defined values. These values apply to all process-proxy kernels, including the defaultLocalProcessProxy
. Note that the typical use-case for this value is to not set--EnterpriseGatewayApp.authorized_users
at the global level, but then restrict access at the kernel level.unauthorized_users
: This process proxy configuration entry can be used to amend--EnterpriseGatewayApp.unauthorized_users
. Any values specified in the config dictionary are added to the globally defined values. As a result, once a user is denied access at the global level, they will always be denied access at the kernel level. These values apply to all process-proxy kernels, including the defaultLocalProcessProxy
.port_range
: This process proxy configuration entry can be used to override--EnterpriseGatewayApp.port_range
. Any values specified in the config dictionary override the globally defined values. These apply to allRemoteProcessProxy
kernels.
Per-kernel environment overrides#
In some cases, it is useful to allow specific values that exist in a kernel.json env
stanza to be
overridden on a per-kernel basis. For example, if the kernel.json supports resource limitations you
may want to allow some requests to have access to more memory or GPUs than another. Enterprise
Gateway enables this capability by honoring environment variables provided in the json request over
those same-named variables in the kernel.json env
stanza.
Environment variables for which this can occur are any variables prefixed with KERNEL_
as well as any variables
listed in the EnterpriseGatewayApp.client_envs
configurable trait (or via
the EG_CLIENT_ENVS
variable). Likewise, environment variables of the Enterprise Gateway
server process listed in the EnterpriseGatewayApp.inherited_envs
configurable trait
(or via the EG_INHERITED_ENVS
variable)
are also available for replacement in the kernel process’ environment.
See Kernel Environment Variables in the Users documentation section for a complete set of recognized KERNEL_
variables.