Struct std::TabularAgent

#include <std/ml/rl.h> source

struct TabularAgent {
    &FloatTensor q;
    unsigned long states;
    unsigned long actions;
    float alpha;
    float gamma;
    float epsilon;
    struct Rng rng;
}

Fields

§&FloatTensor qsource

[states, actions]

§unsigned long statessource

§unsigned long actionssource

§float alphasource

step size

§float gammasource

discount

§float epsilonsource

exploration (change it to decay)

§struct Rng rngsource

Methods

§unsigned long act(unsigned long state)source · definition

epsilon-greedy

§unsigned long greedy(unsigned long state) constsource · definition

§

float q_learning_update(
    unsigned long s,
    unsigned long act,
    float r,
    unsigned long s2,
    int terminal,
)

source · definition

Each returns the TD error. 'terminal' drops the bootstrap term. Q-learning (off-policy): target r + gamma max_a' Q(s', a').

§

float sarsa_update(
    unsigned long s,
    unsigned long act,
    float r,
    unsigned long s2,
    unsigned long a2,
    int terminal,
)

source · definition

SARSA (on-policy): target r + gamma Q(s', a') for the a' actually taken next.

§

float expected_sarsa_update(
    unsigned long s,
    unsigned long act,
    float r,
    unsigned long s2,
    int terminal,
)

source · definition

Expected SARSA: target r + gamma E_pi[Q(s', .)] under the epsilon-greedy policy.

§void drop()source · definition

§void free() consumesource · definition

= drop()

§

static struct TabularAgent new(
    unsigned long states,
    unsigned long actions,
    float alpha,
    float gamma,
    float epsilon,
    unsigned long seed,
)

source · definition