Los Altos was home to the Ohlone people for thousands of years prior to the arrival of the first Europeans. The Ohlone were hunter gatherers and in some ways harvesters. Their diet had wide seasonal variety that included crushed acorns, nuts, grass seeds, berries, wild game, birds, fish and shellfish. These foods were abundant before European arrival and the Ohlone’s maintained some foods through some active management. Waterfowl were the most important birds in the people's diet, which were captured with nets and decoys. In general, along the bayshore and valleys, the Ohlone constructed dome-shaped houses of woven or bundled mats of tules, 6 to 20 feet in diameter. In hills where Redwood trees were accessible, they built conical houses from Redwood bark attached to a frame of wood. One of the main village buildings, the sweat lodge, was low into the ground, its walls made of earth and roof of earth and brush. They built boats of Tule to navigate on the bays propelled by double-bladed paddles.

Generally, men did not wear clothing in warm weather. In cold weather, they would use animal skin capes or feather capes. Women commonly wore deerskin aprons, tule skirts, or shredded bark skirts. On cool days, they also wore animal skin capes. Both men and women wore necklaces, shell beads and abalone pendants, and bone wood earrings with shells and beads.

Spanish explorers were the first Europeans to visit the coastline of what is now California, Juan Rodriguez Cabrillo in 1542 followed by Sebastian Vizcaino in 1602. However, none of the early explorers noticed the San Francisco Bay nor made contact with the local Ohlones.  One hundred and seventy-four years would pass before Spanish explorers would  arrive in what is now Los Altos.

Juan Bautista de Anza led a Spanish expedition through Los Altos in 1776 camping in what is now Cupertino on March 25, 1776. De Anza's goal was to establish a land route from Mexico through Alta California to evade the coastal currents and winds that made sailing up the coast very challenging.  Construction of the California mission system soon followed under Father Junipero Serra and the Franciscans. The mission system was the first major effort by Europeans to colonize the California coast and gave Spain a valuable asset. The closest mission to Los Altos, Mission Santa Clara, was dedicated in 1777. The missionaries and settlers introduced European religion, livestock, fruits, vegetables, cattle, horses and ranching into the California region. The  occupation had an immediate and negative impact on the native Ohlone people. The plants and animals that arrived from Europe changed the flora and fauna in the region profoundly.  The mission system ended with the 1833 Act of Secularization, giving land to the native peoples for homes while the churches remained.

After winning independence from Spain in 1821, California became part of the new country of Mexico which encouraged its settlement through land grants to any Catholic native or naturalized citizen of Mexico. Los Altos was included in the  land grant given by Governor Juan Alvarado to San Francisco Presidio soldier Juan Prado Mesa in 1843. Mesa was born in Santa Clara and was a grandson of an original member of de Anza's 1776 expedition. Mesa later served as commander of the Santa Clara Mission Guard. The 7,982-acre “Rancho San Antonio” was bounded by Adobe Creek to the north and Steven's Creek to the south, and included Permanente Creek and present day Los Altos Hills.

American interest in California began with expeditions by John C. Fremont. After the War with Mexico, California was ceded to the United States under the Treaty of Guadalupe Hidalgo signed on February 2, 1848. California was admitted to the Union on September 9, 1850. California's population dramatically increased following statehood, the Gold Rush and westward settlements.

Gradually, the ranchos were divided and sold as smaller ranches for cattle grazing, orchards, and vineyards. Wealthy San Franciscans also bought properties for summer and weekend retreats.

The initial development of Los Altos is credited to Paul Shoup, an executive and later President of the Southern Pacific Railroad, who wanted to link the cities of Palo Alto and Los Gatos by making Los Altos a commuter town on a proposed new rail line.

Attracted by the natural beauty of the land and its proximity to the county seat of San Jose and the cultural attraction of nearby Stanford University, Shoup joined with colleagues to form the Altos Land Company in 1906. Advertising Los Altos as "the loveliest place on the peninsula", San Franciscans were offered free railroad excursions for a day in the country, along with complimentary picnics alongside the tracks in Los Altos.

The first business in the new town was Eschenbreucher's Hardware Store & Post Office, now 316 Main Street. The Los Altos Water Company, Los Altos Building and Loan, Union Land Company and the railroad company all had offices in downtown. In 1909, the two-story Shoup Building at Main and Second streets, housed a grocery store downstairs while upstairs one teacher taught all eight grades of the first public school classes. By 1911, there were 50 homes in Los Altos and a few office buildings and stores on Main Street with twelve steam trains a day stopping at a two-boxcar railway station.

As Paul Shoup envisioned, Los Altos grew and prospered. The business community thrived and orchards gave way to beautiful tree-lined residential streets. Following World War II, Los Altos experienced a boom in home construction and new schools were built in rapid succession to accommodate the expanding student population.

The town leaders in Los Altos, fearing annexation by neighboring cities, realized that incorporation was one way to control zoning and development. After a bitter campaign and an incorporation election, Los Altos became Santa Clara County's 11th city on December 1, 1952. The first mayor of the new city was A. Watson Conner. Minimum residential lot size was established at one-quarter acre.