Demo

The demo/ folder is not part of the hetero package – it was added at the request of reviewers – and it is not installed. Its modules import each other flat (import hetero_utils), so demo/ has to be on the import path; demo_cli and ml / ns are only importable from inside that folder.

Install the extra dependencies first:

pip install -e ".[demo]"

cd demo
python demo_cli.py --dataset speechcommands --Nnet 10 --N 250
python demo_cli.py --dataset mante --stoch stim

--dataset picks the branch: speechcommands and mnist run ml, mante runs ns. --skip_spaced switches from the spaced base-timescale sweep to a single base timescale.

The demos contrast a homogeneous network against heterogeneous ones: the time constants are spread with hetero_utils.sample_taus() at a level set by hetero_utils.set_het_level(), the same simulated activity is then classified by hetero_utils.classifier(), and the scores of the two network families are compared to quantify the benefit of heterogeneity.

Entry point

demo_cli.main()

Machine-learning demos

ml.count_parameters(model)

ml.get_dataset(name='speechcommands', subset='train')

ml.get_bundle(name='emformer', return_feature_extractor=True, return_decoder=False, return_token_processor=False)

ml.collate_fn_mnist(batch, stream=True, permutation=None)

ml.collate_fn_arged(batch, feature_extractor, stream, keep_classes)

ml.get_train_test(dataset_name='speechcommands', seed=1234, stream=True, Nbatch=100, batch_size=32, top_n=10, permute=True, center=False, unit_std=False, normed_trial=True, orthgonalize=False, whiten=True, zero_phase=True)

ml.run_single_net(h=np.sqrt(10), N=10, p=.1, seed=1234, Nbatch=50, dt=1, unit_std=False, center=False, orthgonalize=True, zero_phase=True, filt=False, first_unstream=True, flattening_method='flat', af='tanh', num_snapshot=10, plot=True)

ml.run_multinet(Nnet=20, h='paper', h_is_multiplicative=True, N=250, p=.1, af='tanh', dt=1, dataset_name='speechcommands', Nbatch=50, top_n='all', unit_std=False, center=False, orthgonalize=False, zero_phase=True, whiten=False, normed_trial=True, filt=False, first_unstream=True, flattening_method='flat', pooling_funcs=['go', 'avg', 'snapshot'], golength=10, nsnapshot=10, seed=1234, n_jobs=-1, cv=10, Creg=1, prefix='', plot_uvx=True, plot_man=True, **kwargs)

ml.run_spaced_tau0_multinet(Nnet=20, h='paper', h_is_multiplicative=True, N=250, p=.1, af='tanh', J=1, Ju=1, dt=1, dataset_name='speechcommands', Nbatch=50, top_n=10, unit_std=False, center=False, orthgonalize=False, zero_phase=True, whiten=False, normed_trial=True, filt=False, first_unstream=True, flattening_method='flat', num_tau0=10, pooling_funcs=['go', 'avg', 'snapshot'], golength=10, nsnapshot=10, seed=1234, n_jobs=-1, cv=10, Creg=1, clf_name='logreg', prefix='', plot_uvx=True, plot_man=False, **kwargs)

Neuroscience demos

ns.make_dataset(seq_len, dim_ring=2, stim_period=100, delay_period=[50, 30, 300], decison_period=10, sigma=1, dt=1, n_trials_to_plot=0, seed=1, verbose=False)

ns.correct_targets(u, y)

ns.draw_uyl(dataset, verbose=True, return_label=True)

Draws new samples from the dataset (psychological environment) and adds the go-cue to the first component of stimulus as it is not provided in ngym. See: https://github.com/neurogym/neurogym/issues/279

Note that due to stochasticity of the delay period, the number of samples (fixation + stimulus + delay + go-cue) is not the same between different calls. Thus, we also return the length of each trial.

ns.get_input_channel_names(u)

ns.get_trial_onset_idxs(u, fix_column=1)

ns.get_go_idxs(u, go_column=0)

ns.get_go_onset_idxs(u, go_column=0)

ns.get_go_offset_idxs(u, go_column=0)

ns.get_go_interval(u, arr=None)

ns.get_train_test(dataset, center=False, unit_std=False, normed_trial=True, orthgonalize=False, whiten=False, zero_phase=False)

Generates two independent seqeunce from the dataset, used as the train and test sets. The number of samples in each set is roughly the same for long sequences (to be defined upon instantiation of the dataset object). Stimulus channels are centered using the statistics of the train set.

Returns train and test tuples containing:

• stimulus u (nseq x dim-u)

• labels y (nseq x num-classes)

• go-cue indices idx (nseq)

ns.run_single_net(h='paper', h_is_multiplicative=True, tau0=None, N=250, p=.1, seed=2026, seq_len=200000, dt=1, dim_ring=6, af='tanh', pooling='snapshot', nsnapshot=10, golength=10, unit_std=False, center=False, orthgonalize=True, zero_phase=True, filt=False, first_unstream=True, flattening_method='flat')

ns.run_multinet(seed, Nnet, N, p, af, h, h_is_multiplicative, seq_len, dt, dim_ring, delay_period, stim_period, unit_std, center, orthgonalize, zero_phase, whiten, filt, first_unstream, flattening_method, pooling_funcs, golength, nsnapshot, n_jobs, cv, Creg, prefix, plot_uvx, plot_man, **kwargs)

ns.run_spaced_tau0_multinet(seed, Nnet, N, p, af, J, Ju, h, h_is_multiplicative, seq_len, dt, dim_ring, delay_period, stim_period, num_tau0, pooling_funcs, golength, nsnapshot, n_jobs, cv, Creg, clf_name, prefix, plot_uvx, plot_man, sigma, **kwargs)

Shared helpers

This module contains functions to create network topologies and initialize weights. Currently, only the random connectivity topology is implemented.

hetero_utils.polish(fig, axs, lw=1, tight=True)

hetero_utils.simpleaxis(ax, lw=2)

hetero_utils.get_cax(ax, width=3., dist=.05)

Returns a human-friendly colorbar axis from a given ax, to be used with matplotlib’s colorbar method.

hetero_utils.create_topology(N, N_syn, allow_autapse=False, topology='random', rng=None)

Create sparse connectivity matrix following specified topology rules.

Parameters:

• N (int) – Number of neurons

• N_syn (int) – Number of synapses (determines sparsity)

• allow_autapse (bool, optional) – Whether to allow self-connections (default False)

• topology (str, optional) – Type of topology to create (default ‘random’) Currently supported: ‘random’

• rng (numpy.random.Generator, optional) – Random number generator instance

Returns:

scipy.sparse.csr_matrix – Sparse adjacency matrix of shape (N, N)

hetero_utils.intialize_weights(N_input, N, N_output, p, f, mu_e, sigma_0, allow_autapse=False, return_adj=True, rng=None)

Creates three weight matrices:

• the feedforward weights w_ff mapping inputs to neurons,

• the recurrent weights w between neurons, and

• an empty feedback weight array w_fb (a placeholder for future use).

The recurrent weights are initialized to ensure excitatory/inhibitory balance, whereas the feedforward weights are initialized from a centered normal distribution.

Parameters:

• N_input (int) – Number of input dimensions

• N (int) – Number of neurons

• N_output (int) – Number of output dimensions

• p (float) – Connection probability

• f (float) – Fraction of excitatory neurons

• mu_e (float) – Mean excitatory weight. The mean inhibitory weight is computed based on f to ensure E/I balance.

• sigma_0 (float) – Recurrent weight standard deviation (same for both excitatory and inhibitory)

• return_adj (bool, optional) – Whether to return adjacency matrices. Default is True.

• rng (numpy.random.Generator, optional) – Random number generator instance

Returns:

• w_ff (ndarray) – Input weights of shape (N, N_input)

• w (ndarray) – Recurrent weights of shape (N, N)

• w_fb (ndarray) – Feedback weights of shape (N, N_output)

Note

• The recurrent weights are scaled by \(1/\sqrt{Np}\) to ensure the expected recurrent input variance, independent of the network size and connection probability, is equal to sigma_0.

• The feedforward weights are scaled by \(1/\sqrt{dim_{input}}\) to ensure that feedforward input variance on each neuron is equal to 1 independent of the input dimension.

• The feedforward and recurrent gains (\(J_u\) and \(J\), respectively), will be applied later during integrating the dynamical system.

hetero_utils.get_spaced_hom_taus(lens, dt=1, num=10)

We select taus smaller than the minimal length and taus between the minimal and maximal length.

hetero_utils.extract_base_tau(us, dt, first_unstream=True, sample_lengths=None, center=True, filt=True, flattening_method='flat', savepath='', savename=None)

hetero_utils.sample_taus(size, mu, std, distr='lognormal', clip_below=False, rng=None, tol_prcntg=-1, verbose=False, savepath='.', savename=None)

The given mu and std are the desired mean and standard deviation and are different from the parameters passed to rng. It can also handle the case of std=0, by returning a constant vector of the desired size.

hetero_utils.generate_net(N_input, N=50, p=.1, f=0.8, mu_e=1, sigma_0=1, allow_autapse=False, savepath='.', savename='net', **kwargs)

hetero_utils.activation_func(x, af, x0=1., thr=0, **kwargs)

Apply a specified activation function to input data with optional scaling and thresholding. The valid activation fucntions are:

• ‘lin’: linear

• ‘relu’: rectified linear unit

• ‘caplin’ : same as ‘lin’ but capped between 0 and 1

• ‘caprelu’: same as ‘relu’ but capped between 0 and 1

• ‘capquad’: quadratic function capped between 0 and 1

• ‘capcube’: cubic function capped between 0 and 1

• ‘capcube’: cubic function capped between 0 and 1

• ‘capexp’: exponential function capped between 0 and 1

• ‘tanh’: tangent hyperbolic

• symsig: an even sigmoidal function centered at t=0 with range between -1 and 1.

• ‘possig’: sigmoidal function centered at y=0.5 with range between 0 and 1 .

• ‘possig_plus’: same as ‘possig’ but only for values larger than threshold and zero otherwise.

• ‘bell’ : non-normalized Gaussian activation function

• ‘camel’: a gaussian multiplied by a pure quadratic function, resulting in two identical bumps

  at (±1, 1).

• ‘switch’: a binary activation. Zero for values smaller than thr and 1 otherwise.

Parameters:

• x (float or ndarray) – Input array or value to be transformed

• af (str) – Name of the activation function to apply (e.g., ‘relu’, ‘tanh’, ‘possig’, etc.)

• x0 (float) – Scaling factor for the input (default: 1.0)

• thr (float) – Threshold (bias) value to shift the input (default: 0)

• kwargs (dict) – Additional keyword arguments for specific activation functions

Returns:

float or ndarray – The result of applying the chosen activation function to the input

hetero_utils.activation_func_deriv(x, af, x0=1., thr=0, **kwargs)

Apply a specified activation function to input data with optional scaling and thresholding. The valid activation fucntions are:

• ‘lin’: linear

• ‘relu’: rectified linear unit

• ‘caplin’ : same as ‘lin’ but capped between 0 and 1

• ‘caprelu’: same as ‘relu’ but capped between 0 and 1

• ‘capquad’: quadratic function capped between 0 and 1

• ‘capcube’: cubic function capped between 0 and 1

• ‘capcube’: cubic function capped between 0 and 1

• ‘capexp’: exponential function capped between 0 and 1

• ‘tanh’: tangent hyperbolic

• symsig: an even sigmoidal function centered at t=0 with range between -1 and 1.

• ‘possig’: sigmoidal function centered at y=0.5 with range between 0 and 1 .

• ‘possig_plus’: same as ‘possig’ but only for values larger than threshold and zero otherwise.

• ‘bell’ : non-normalized Gaussian activation function

• ‘camel’: a gaussian multiplied by a pure quadratic function, resulting in two identical bumps

  at (±1, 1).

• ‘switch’: a binary activation. Zero for values smaller than thr and 1 otherwise.

Parameters:

• x (float or ndarray) – Input array or value to be transformed

• af (str) – Name of the activation function to apply (e.g., ‘relu’, ‘tanh’, ‘possig’, etc.)

• x0 (float) – Scaling factor for the input (default: 1.0)

• thr (float) – Threshold (bias) value to shift the input (default: 0)

• kwargs (dict) – Additional keyword arguments for specific activation functions

Returns:

float or ndarray – The result of applying the chosen activation function to the input

hetero_utils.integrate(u, taus, A_adj, w_in, w_rec, dt, J=1., Ju=1., af='possig', reset_dim=0, u_reset=None)

hetero_utils.classifier(name='logreg', cv=10, random_state=0, solver='newton-cg', max_iter=200, scoring='accuracy', C=1, **kwargs)

hetero_utils.unstream(quant, Tsample, Nsample)

hetero_utils.plot_taus(tau_hom, tau_het, savepath, savename)

hetero_utils.plot_dataset_hist(y, savepath='.', savename='')

hetero_utils.plot_u_stream(u, lengths, input_labels=None, figsize=(8, 5), tmin=0, tmax=500, savepath='.', savename='')

hetero_utils.plot_uvx_stream(u, v, x, labels=['input', 'voltage', 'activation'], cmaps=['rainbow', 'PRGn', 'Grays'], input_labels=[], figsize=(8, 8), tmin=0, tmax=500, lengths=None, savepath='.', savename='')

hetero_utils.plot_manifold(u, v, x, y, lengths, manifold='pca', standarized=True, figsize=(8, 3), savepath='.', savename='', seed=2026)

Gets stream-version of inputs u, voltages v, and activations x, as well as the class label of each sample y, breaks the streamed arrays into a flatten array containing fearues of a single sample, and runs a manifold algorithm for cluster identificaiton. Points are colored by their class. Array of classes should be ordinal numbers.

hetero_utils.plot_score_summary(score_trn_hom, score_trn_het, score_tst_hom, score_tst_het, savepath, savename, chance_score=None, pooling_funcs=None, Cregs=None)

hetero_utils.plot_score_summary_of_spaced_hom(score_trn_hom, score_trn_het, score_tst_hom, score_tst_het, spaced_taus_hom, pooling_funcs, Cregs, savepath, savename, chance_score=None)

hetero_utils.get_snapshot(lengths, nsnapshot=10, arr=None, flatten=False)

hetero_utils.pool(arr, trial_lengths, func='max', n=-1, nsnapshot=10, golength=10, flatten=True)

hetero_utils.get_trial_start_end(lens)

hetero_utils.get_trial_stack(arr, lens)

hetero_utils.get_statistics(us, lens, per_length=True)

hetero_utils.set_het_level(value='paper', lens=None)

hetero_utils.plot_taus_summary(het_taus, hom_taus, savepath, savename='tau_summary')

hetero_utils.collect_taus(savepath, Ntau0, Nnet)

hetero_utils.collect_scores(savepath, Ntau0, Nnet, Npool, NCregs=1)

hetero_utils.sample_integrate_classify_save(dt, tau0, h_value, h_is_multip, N, p, J, Ju, us_trn, us_tst, lens_trn, lens_tst, ys_trn, ys_tst, pooling_funcs, pooling_kwargs, af, cv, Creg, clf_name, prefix, savepath, plot_uvx=True, plot_man=True, net_idx=0, tau0_idx=0, reset_dim=0, u_reset=None)