Pulses, Bursts of Activity

danger

THIS NEEDS REFACTORING

A pulse denotes a short burst of energy or a rapid transient change in the state of something, often seen in waveforms like sound waves, light waves, or electrical currents. The concept of a pulse is pivotal in understanding how energy is transmitted through different mediums.

Take, for example, a sound wave. When you clap your hands, you create a pressure wave in the air. This wave is a form of a pulse—a quick, transient disturbance that travels through the air, carrying energy from the point of the clap to your ears. The wave's energy is concentrated in its pulse, and it diminishes as it spreads out and loses energy to the surrounding environment.

Pulses create a localized decrease in entropy. This is analogous to the pulses in our platform, which can be AI models, transformation codes, or human inputs. Just as a physical pulse imposes a temporary order in a system by aligning particles or waves in a coherent manner, our pulses bring order to the data within the nebula of our platform. They transform raw, unstructured, or chaotic data into a more organized and useful form. In this way, the entropy of the data – its randomness and disorder – is reduced, making the information more accessible and actionable.

Each pulse serves as a moment of organization and clarity within the vast and chaotic universe of data. These pulses, whether they originate from sophisticated AI algorithms, carefully crafted code, or the strategic interventions of human experts, act as agents of order, transforming the nebula of raw data into a cosmos of structured, meaningful information.

So, while in the grand tapestry of the universe, entropy tends to increase, in the microcosm of our platform, each pulse is a step towards creating a more ordered, understandable, and valuable data universe. It's a little bit like being cosmic librarians, sorting and categorizing the stars and galaxies of data into a system that makes sense – and all of this is done with such ease that we had to introduce some complexity by naming our tasks 'pulses', just to keep things interesting!

Types

Pulses are the unit of execution. Gates, AI models, Packagers, and loaders are all Pulses.

Input	Type	Output
Defined Format	Gate	Knowledge Graph
Knowledge Graph	AI Model	Knowledge Graph
Knowledge Graph	Packager	Defined Format

Any Pulse used in a workflow is implemented as a Flow in Metaflow.