A variable- (also changeable-, electronic-, or dynamic-) message sign or message board, often abbreviated VMS, VMB, CMS, or DMS, and in the UK known as a matrix sign, is an electronic traffic sign often used on roadways to give travelers information about special events. Such signs warn of traffic congestion, accidents, incidents such as terrorist attacks, Amber/Silver/Blue Alerts, roadwork zones, or speed limits on a specific highway segment. In urban areas, VMS are used within parking guidance and information systems to guide drivers to available car parking spaces. They may also ask vehicles to take alternative routes, limit travel speed, warn of duration and location of the incidents, inform of the traffic conditions, or display general public safety messages. == History == VMS systems were deployed at least as early as the 1950s on the New Jersey Turnpike. The road's signs of that period, and up to around 2012, were capable of displaying a few messages in neon, all oriented around warning drivers to slow down: "REDUCE SPEED", followed by a warning of either construction, accident, congestion, ice, snow, or fog at a certain distance ahead. The New Jersey Turnpike Authority replaced those signs (along with 1990s-vintage dot-matrix VMS systems along the Garden State Parkway) with more flexible electronic signs between 2010 and 2016. The current VMS systems are largely deployed on freeways, trunk highways, or in work zones. On the interchange of I-5 and SR 120 in San Joaquin County, California, an automated visibility and speed warning system was installed in 1996 to warn traffic of reduced visibility due to fog (where tule fog is a common problem in the winter), and of slow or stopped traffic. Message Signs were deployed in Ontario during the 1990s and are now being upgraded on 400-series highways as well as two pilot secondary highways in northeastern Ontario. == Technologies and types == Early variable message signs included static signs with words that would illuminate (often using neon tubing) indicating the type of incident that occurred, or signs that used rotating prisms (trilons) to change the message being displayed. These were later replaced by dot matrix displays typically using eggcrate, fiber optic, or flip-disc technology, which were capable of displaying a much wider range of messages than earlier static variable message signs. Since the late 1990s, the most common technology used in new installations for variable message signs are LED displays. In recent years, some newer LED variable message signs have the ability to display colored text and graphics. Dot-matrix variable message signs are divided into three subgroups: character matrix, row matrix, and full matrix. In a character matrix VMS, each character is given its own matrix with equal horizontal spacing between them, typically with two or three rows of characters. In a full matrix VMS, the entire sign is a single large dot matrix display, allowing the display of different fonts and graphics. A row matrix VMS is a hybrid of the two types, divided into two or three rows like a character matrix display, except each row is a single long dot matrix display instead of being split per character horizontally. Overhead variable message signs are today available in three form factors: front access, rear access, and walk-in. In a front access variable message sign, maintenance is performed by lifting the sign open from the front. Most smaller VMS are of the front access form factor, and are typically installed today on major arterials. The rear access form factor is similar to the front access form factor, except that maintenance is performed from the rear of the sign, and are commonly used for medium-sized dynamic message signs installed along the roadside of freeways (instead of overhead). The walk-in form factor is a more recent introduction, where maintenance on the sign is performed from the inside of the sign. A key advantage of the walk-in form factor is that lane closures are generally not required to perform maintenance on the sign. Most of the largest VMS units installed today are walk-in units, and are typically installed overhead on freeways. The NJ Turnpike Authority counts five unique types of variable message signs under its jurisdiction, at least one of which has been replaced by newer signs. They are: "REDUCE SPEED" neon signs (1950s-2010, obsolete, have now been replaced). "Changeable message signs" (trilon/ rotating-drum signs that can be used for closing roads or moving traffic to other roadways). Electronic VMS: signs with remotely controlled messages displayed on them; the messages are sent from the State Traffic Management Center, updating the signs automatically. Variable speed limit signs - used for varying the posted speed limits within work zones and in emergencies. Portable VMS: movable "electronic VMS". A portable VMS has much the same characteristics as a fixed electronic VMS, but can be moved from location to location as the need dictates. == Usage == Early models required an operator to be physically present when programming a message, whereas newer models may be reprogrammed remotely via a wired or wireless network or cellphone connection. A complete message on a panel generally includes a problem statement indicating incident, roadwork, stalled vehicle etc.; a location statement indicating where the incident is located; an effect statement indicating lane closure, delay, etc. and an action statement giving suggestion what to do traffic conditions ahead. These signs are also used for Amber alert messages, and in some states, Silver and Blue Alert messages. In some places, VMSes are set up with permanent, semi-static displays indicating predicted travel times to important traffic destinations such as major cities or interchanges along the route of a highway. Typical messages provide the following information: Promotional messages about services provided by a road authority during non-critical hours, such as carpooling efforts, travelers' information stations and 5-1-1 lines Crashes, including vehicle spin-out or rollover Road Works Incidents affecting normal traffic flow in a lane or on shoulders Non-recurring congestion, often a residual effect of cleared crash Closures of an entire road, e.g. over a mountain pass in winter. Exit ramp closures Debris on roadway Vehicle fires Wildfires Short-term maintenance or construction lasting less than three days Pavement failure alerts AMBER, Silver, and Blue Alerts, as well as weather warnings via the warning infrastructure of NOAA Weather Radio's SAME system Travel times Variable speed limits Car park occupancy levels speed sign, for recommending a speed to approach the next traffic light in its green phase. The information comes from a variety of traffic monitoring and surveillance systems. It is expected that by providing real-time information on special events on the oncoming road, VMS can improve motorists' route selection, reduce travel time, mitigate the severity and duration of incidents and improve the performance of the transportation network. === United Kingdom === Do not enter the motorway when the red lamps are flashing in pairs from side to side. On 27 March 1972, the first motorway computer-controlled warning lights in the UK, with 59 miles on the M6 from Broughton, Lancashire to Barthomley, on the Cheshire boundary, and 26 miles on the M62 east of Whitefield, was switched on by Michael Heseltine and Charles Legh Shuldham Cornwall-Legh, 5th Baron Grey of Codnor at the headquarters of Cheshire Constabulary on Nuns Road. It was centred at a police computer centre at Westhoughton, that connected to police stations in Preston and Chester. The Chester site was soon be connected to the M53 and M57. Four other regional computer centres would be opened at Perry Barr near the M6, Scratchwood near the M1, at Hook near the M3, and at Almondsbury near the M4. Most British motorways would be covered by 1975. The system was designed by GEC and had taken five years to design. == Safety messages for drivers == Increasingly, signs have been used to remind drivers to buckle seat belts ("Click It or Ticket"), obey the speed limit, and stay off the road if impaired ("Drive sober or get pulled over"). In a federal study, a slight majority of drivers reported that public safety messages on dynamic message signs impacted their driving behaviors. The Ohio Department of Transportation began using humorous dynamic message signs in 2015, perplexing some drivers. Examples of humorous signs seen in New Jersey, Arizona, Texas, Pennsylvania, Delaware, Iowa, New York, Minnesota and Ohio include: "Hold on to your butts. Help prevent forest fires." "We'll be blunt. Don't drive high." "Visiting in-laws? Slow down, get there late." "Only sparklers should be lit." and “Don’t drive Star Spangled hammered." (for Fourth of July) "Hocus pocus – drive with focus." and "Slow down in work zones - my mummy works here." (f
Region Based Convolutional Neural Networks
Region-based Convolutional Neural Networks (R-CNN) are a family of machine learning models for computer vision, and specifically object detection and localization. The original goal of R-CNN was to take an input image and produce a set of bounding boxes as output, where each bounding box contains an object and also the category (e.g. car or pedestrian) of the object. In general, R-CNN architectures perform selective search over feature maps outputted by a CNN. R-CNN has been extended to perform other computer vision tasks, such as: tracking objects from a drone-mounted camera, locating text in an image, and enabling object detection in Google Lens. Mask R-CNN is also one of seven tasks in the MLPerf Training Benchmark, which is a competition to speed up the training of neural networks. == History == The following covers some of the versions of R-CNN that have been developed. November 2013: R-CNN. April 2015: Fast R-CNN. June 2015: Faster R-CNN. March 2017: Mask R-CNN. December 2017: Cascade R-CNN is trained with increasing Intersection over Union (IoU, also known as the Jaccard index) thresholds, making each stage more selective against nearby false positives. June 2019: Mesh R-CNN adds the ability to generate a 3D mesh from a 2D image. == Architecture == For review articles see. === Selective search === Given an image (or an image-like feature map), selective search (also called Hierarchical Grouping) first segments the image by the algorithm in (Felzenszwalb and Huttenlocher, 2004), then performs the following: Input: (colour) image Output: Set of object location hypotheses L Segment image into initial regions R = {r1, ..., rn} using Felzenszwalb and Huttenlocher (2004) Initialise similarity set S = ∅ foreach Neighbouring region pair (ri, rj) do Calculate similarity s(ri, rj) S = S ∪ s(ri, rj) while S ≠ ∅ do Get highest similarity s(ri, rj) = max(S) Merge corresponding regions rt = ri ∪ rj Remove similarities regarding ri: S = S \ s(ri, r∗) Remove similarities regarding rj: S = S \ s(r∗, rj) Calculate similarity set St between rt and its neighbours S = S ∪ St R = R ∪ rt Extract object location boxes L from all regions in R === R-CNN === With R-CNN, prediction follows a two-step process. A preprocessing selective search step generates a large set of candidate objects (typically as many as 2000), known as regions of interest (ROI). These are forwarded to a CNN, which predicts an object class score and bounding box estimate, independently for each ROI. Importantly, the ROIs are heavily filtered to remove excess candidates. This is achieved using two mechanism. Filtering begins by removing ROIs assigned to the background category. This is a specialized category, which is scored by the CNN alongside other categories. An unfortunate reality is that remaining ROIs typically suffer from heavy duplication. Namely, multiple ROIs that cover same objects in the image are all assigned non-background categories. This is resolved by a heuristic non-maximum suppression (NMS) step. === Fast R-CNN === While the original R-CNN independently computed the neural network features on each of as many as two thousand regions of interest, Fast R-CNN runs the neural network once on the whole image. At the end of the network is a ROIPooling module, which slices out each ROI from the network's output tensor, reshapes it, and classifies it. As in the original R-CNN, the Fast R-CNN uses selective search to generate its region proposals. === Faster R-CNN === While Fast R-CNN used selective search to generate ROIs, Faster R-CNN integrates the ROI generation into the neural network itself. === Mask R-CNN === While previous versions of R-CNN focused on object detections, Mask R-CNN adds instance segmentation. Mask R-CNN also replaced ROIPooling with a new method called ROIAlign, which can represent fractions of a pixel.
Age Of
Age Of is the eighth studio album by American electronic producer Oneohtrix Point Never, released on June 1, 2018, on Warp Records. Recorded over two years, it is the first Oneohtrix Point Never album to prominently feature Daniel Lopatin's own vocals. The album was accompanied by the MYRIAD tour, which premiered as a "conceptual concertscape" in 2018 at the Park Avenue Armory and ended its run in 2019. It features contributions from James Blake (who additionally produced and mixed the album), Anohni, Prurient, Kelsey Lu and Eli Keszler. The artwork, which employs Jim Shaw's "The Great Whatsit" as a central image, was designed by David Rudnick. While not entering the official United States Billboard 200 chart, it peaked at number 59 on the magazine's Top Current Albums chart. == Background == Lopatin produced Age Of in parts of a two-year period, during which he was also producing for other artists, including Anohni, FKA Twigs, Iggy Pop, and David Byrne. After composing the soundtrack for the Safdie Brothers' 2017 film Good Time, Lopatin moved to an Airbnb lodge in South Central Massachusetts, derived from his aspiration to live out the modern cliche of musicians moving to the woods to record albums; the eerie atmosphere in the lodge at nighttime influenced his desire to make "weird, little nightmare ballads". In addition to Lopatin's own singing, the album also features vocal performances from Anohni and Prurient, while instrumentalists Kelsey Lu and Eli Keszler contribute to several tracks. When the record was nearly finished, Lopatin reached out to musician James Blake to contribute to the mixing process, eventually traveling to Los Angeles to complete the album. The track "The Station" was originally composed as a demo for R&B singer Usher which was ultimately not used. On July 9, 2018, Lopatin released the original topline (vocal melody) demo for The Station through Sendspace. The track "Toys 2" imagines a theoretical sequel to the 1992 film Toys where actor Robin Williams' image has been recreated with CGI (as his will specifically forbade any usage of his image after his death), and pokes fun at the common electronic music trope of composing a soundtrack to a theoretical film (which Lopatin described as "horribly cliché"). == Concept and MYRIAD == Influences on Age Of included Stanley Kubrick's 1968 film 2001: A Space Odyssey, which inspired the narrative of the album's accompanying performance installation and tour MYRIAD, as well as William Strauss's The Fourth Turning, a favorite book of former White House Chief Strategist Steve Bannon, which Lopatin described as "insidious, like the voice of a computer insisting on the truth about history without any sensitivity given to how complex and non-linear systems might be"; Lopatin was subsequently inspired to "[use] that sort of taxonomy as a kind of farce to then create these little frameworks for understanding". Other inspirations included the writings of the 1990s multidisciplinary collective Cybernetic Culture Research Unit and the works of singer-songwriters such as Bruce Cockburn, Bob Dylan, and Paul Simon. Around the time Lopatin began finalizing Age Of in his Airbnb lodge, he began working on the concept for MYRIAD, a conceptual concert performance which premiered at Park Avenue Armory. He described the concept as a four-part "epochal song cycle" showcasing the idiocy of previous generations of living organisms. The loose story concerns a group of artificial intelligences near the end of time named a "Limitless Living Informational Intelligence" (represented in the MYRIAD logo as nine squares) which, for leisurely purposes, attempt to replicate the cultures and behaviors of the previously existent human species. It does this by determining an "average" of human experiences through the species' "recorded output", and does so through imperfect, heuristic techniques. The show was consequently divided into four sections, each representing an epoch of the cycle concept loosely inspired by the Strauss–Howe generational theory: the Age of Ecco, the Age of Harvest, the Age of Excess, and the Age of Bondage. Ecco is "a phase of pre-evolutionary ignorance", Harvest is "living in agrarian harmony with the world", Excess is "the age of unchecked industrial ambition", and Bondage is "an era of engorgement, wherein "we keep making more and more shit until there's no space left." MYRIAD mainly featured "three-hundred pound sculptures that hang from the ceiling like kebabs that secrete ooze", and a full ensemble that toured to perform songs from Age Of, including Eli Keszler, Kelly Moran and Aaron David Ross. The sculptures, as well as the visuals displayed on five polygon panels, were created by frequent Oneohtrix Point Never collaborator Nate Boyce. Initially, Lopatin planned for each of the album's four epoches to be represented by fragrances, the more noisy epochs being pleasant to the nose to make a "weird dissonance". However, due to lack of time and resources, that part of the plan was scrapped. == Composition == Whereas previous Oneohtrix Point Never albums followed musical styles from only distinctive eras, Age Of is the first album by Lopatin to incorporate elements of unique genres from a variety of periods, hence the "incompleteness" of its title according to reviewer Heather Phares, and his first pop-song-oriented release since his work for Ford & Lopatin. The sound palettes it uses are those from a variety of styles such as chamber pop, "android"-like folk and country music, yacht rock, smooth jazz, R&B, Future-style soul, black metal, new age, and stadium pop, as well as post-industrial sounds on tracks like "Warning", "We'll Take It" and "Same", and, in particular, baroque music and medieval music on the opening title track, "Age Of". Critics also noted elements of Lopatin's past discography being present on Age Of. The instrumentation of Age Of is made up of MIDI harpsichords, guitars, pianos, brass and vocals, as well as Lopatin's trademark unorthodox sound design, samples and synth presets. The LP's use of the harpsichord shows its similarities "with Eastern instruments such as the koto and with rapid-fire electronic melodies", wrote Phares. == Critical reception == Age Of was critically well-received upon its distribution. Some reviewers praised the album's use of collaborators. Reviewing the album for AllMusic, Heather Phares called Age Of a "landmark work" for Lopatin. She praised it as his "widest-ranging" release, elaborating that he "matches the album's ambition with plenty of emotion" and "gives his music exciting new shapes." Ross Devlin of The Skinny, in a five-star review of the record, also highlighted the album's amount of ambition, particularly the "wealth of exquisitely baroque moments, exploring history as a pliable, multi-dimensional rift", that gave it "exceptional sonic depth". The Observer praised Age Of for continuing the "off-kilter composition and unexpected instrumentation" of Lopatin's previous releases, and critic Matt McDermott highlighted that the producer increased his musical range with the record: "It's a dizzying trip meant to shore up Lopatin's status as an avant-garde auteur while aiding his forays into mainstream pop culture." Age Of was ranked the 15th best release of the year in The Wire magazine's annual critics' poll. == Track listing == Notes "Myriad Industries" is stylized as "myriad.industries". Sample credits "Age Of" contains a sample of "Blow the Wind" by Jocelyn Pook. "Manifold" contains a sample from "Overture (Ararat the Border Crossing)" by Tayfun Erdem; and a sample from "Venice Beach in Winter" (listed in the liner notes as "a keyboard sample from Reharmonization") by Julian Bradley. "Myriad Industries" contains a sample of "EchoSpace" by Gil Trythall. == Accolades == == Personnel == Daniel Lopatin – production, lead vocals, album art, design James Blake – additional production, mixing, keyboards Gabriel Schuman, Joshua Smith and Evan Sutton – assistance Greg Calbi – mastering David Rudnick – album art, design Prurient – vocals Kelsey Lu – keyboards Anohni – vocals Eli Keszler – drums Shaun Trujillo – words == Charts ==
Rifts (role-playing game)
Rifts is a multi-genre role-playing game created by Kevin Siembieda in August 1990 and published continuously by Palladium Books since then. It takes place in a post-apocalyptic future, deriving elements from cyberpunk, science fiction, fantasy, horror, western, mythology and many other genres. Rifts serves as a cross-over environment for a variety of other Palladium games with different universes connected through "rifts" on Earth that lead to different spaces, times, and realities that Palladium calls the "Rifts Megaverse". Rifts describes itself as an "advanced" role-playing game and not an introduction for those new to the concept. Palladium continues to publish books for the Rifts series, with about 80 books published between 1990 and 2011. Rifts Ultimate Edition was released in August 2005 and designed to update the game with Palladium's incremental changes to its system, changes in the game world, and additional information and character types. The web site is quick to point out that this is not a second edition but an improvement and expansion of the original role playing game. == Background == The RPG had the tentative title Boomers, named after the original name for the Glitter Boy power armor until Kevin Siembieda changed the name after finding out it was in use for Bubblegum Crisis. == Setting == The Rifts world is Earth, but hundreds of years into the future. Ley lines, lines of magic energy, criss-cross the earth forming supernatural geographic areas such as the Bermuda Triangle. Points where Ley Lines intersect, called a nexus, are places of powerful magic, such as the Pyramids of Giza and Stonehenge. If a Ley Line nexus energy surges or is purposely activated, the fabric of space and time can be torn, creating a rift - a hole in space-time leading to another place, time, or dimension. Ley lines contain magical energy called Potential Psychic Energy (PPE), which is found in various places, objects, and animals and is particularly strong in children. An adult's level of PPE can vary based on other factors. PPE also allows Psionics which uses energy known as Inner Strength Points or ISP. Psychic phenomenon (more commonly called psionics) can also vary from individuals, ranging from none at all to Master level abilities. Psychic abilities can manifest in virtually any way imaginable. Some psychics develop differently, such as psi-stalkers; human mutants that feed on psychic energy. === Earth === Rifts begins with two future-historical premises: first, a golden age of humanity occurs, with tremendous advances in science, technology, military, and society. Humanity as a whole is at peace as a majority of Earth's nations decide to cease world war and begin to share ideas and technology freely. Much of the Solar System is conquered, humanity's wars will end, and harmony will reign. This golden age is followed by an unknown cause near the winter solstice and a rare planetary alignment, causing a disaster that cascades into tremendous destruction via a ripple effect. The cataclysm begins with unprecedented storms, earthquakes, tsunamis, and volcanic eruptions, which kill millions of people. The Ley Line networks that crisscross the globe are energized, causing rifts to open both on Earth and throughout the Megaverse. For hundreds of years after the holocaust, many creatures, both mythical beasts and aliens, come through the Rifts to wreak havoc. The old world gone, a new Dark Age dawns and humanity's shrinking population is reduced, due to catastrophe and domestic failure, immeasurably. This period is covered in Palladium's Rifts Chaos Earth spin-off series. Rifts initially takes place in 101 P.A. (equivalent to the year 2387) 289 years after this event. The "Post-Apocalypse" calendar was established by the formation of the Coalition States in 2286. By this time, most of the disasters have quieted down, though Earth is still bathed in PPE. The planet's mystical energy has attracted aliens from other dimensions, who continue to arrive through the Rifts both accidentally and deliberately. The humanoid creatures that arrive on Earth are referred to as Dimensional Beings (called D-Bees). Some resemble familiar fantasy races, such as elves and dwarfs, while others were created specifically for the game setting. Non-humanoid creatures have also arrived, including monstrous creatures and mystical demons. To cope with these natural, supernatural, and alien menaces, the human race has adapted in a variety of ways, many of them borrowed from the technological developments of the lost Golden Age. Powered armor suits and giant vehicles are frequently used to combat the dangers of Rifts, but more invasive augmentation is common. This has three basic categories: "Juicers" augment themselves chemically, the "Borgs" augment themselves mechanically, and "Crazies" use performance-enhancing brain implants. All such augmentations boost strength, speed, endurance, and dexterity to superhuman levels. However, all come at great cost. Chemicals cause the body to wear out faster, decreasing life span to a few years. Mechanical Borg augmentation causes a loss of humanity when those with multiple limb and organ replacements become more machine than human. Brain implants cause mental instability ranging from mild phobias to crippling neurosis or psychosis. ==== North America ==== The strongest power in North America is the Coalition States (CS), which is based in the arcological city of Chi-Town and lays claim to northern Illinois, all of Iowa, the Texas Panhandle, Missouri, and the eastern half of Ontario, Canada. The second greatest power is Free Quebec, a former Coalition State that seceded following a civil war with the other Coalition States. Mexico is ruled by a group of vampire kingdoms, who treat humans as little more than food. North of the Rio Grande, west of Texas and roaming most of the American Southwest are large nomadic bands/tribes of bandits who collectively form the Pecos Empire, consisting of El Paso, Los Alamos, and Houstown. Much of the western United States has more or less willingly reverted to a mix of modern and past technology akin to the Wild West. The Royal Canadian Mounted Police managed to survive the great cataclysm, though Canada itself did not. The Mounties have become an independent law enforcement force called the Tundra Rangers, patrolling the northern wilderness. The Midwest, both upper and central, is home to most of North America's population. The Manistique Imperium and Northern Gun in Michigan's Upper Peninsula, both Coalition allies, are among the largest weapons manufacturing areas on the continent. New Lazlo is one of the largest cities in Michigan's southern portion. Chillicothe in Missouri is a large supplier of Coalition food processing and growing. Missouri's southern half, home to the city-states of Whykin (Poplar Bluff) and Kingsdale (West Plains) are in constant opposition to the CS and claim independence. Arkansas is home to the independent CS ally El Dorado. Southern Illinois and the Ohio Valley is home to the Federation of Magic. Also in the Ohio Valley is Psyscape, a city-state founded by psychics. Tolkeen was a major city in the former Minneapolis region in early Rifts books; the city welcomed users of magic. A military campaign made by the Coalition States (which is the primary event of 109 PA) resulted in the magic-user kingdom being wiped off the map. In the Northeast, the city-state of Lazlo, named after supernatural researcher and writer Victor Lazlo, was built upon the ruins of Toronto. This major center of civilization is well known as a melting pot of humans, D-Bees and other beings, and is the home of Techno-Wizardry. Mad Haven is the name given to the ruins of Manhattan; tectonic forces during the cataclysm have moved it into the coast, creating a peninsula. It is seen by most denizens of Rifts Earth as a refuge of demons and madness. ==== South America ==== The return of Atlantis caused the Amazon River basin to flood most of western South America, giving it the nickname The Land of a Thousand Islands. The Empire of the Sun, consisting of Cuzco, Nazca, Arequipa and Lima, created a wide range of technology and magic, including magic derived from the Nazca lines. In Argentina, the Silver River Republics of Cordoba (the South American Chi-Town), Santiago (one of the most tolerant human nations on Rifts Earth), Achilles (a nation founded by mutants), and New Babylon, a nation where humans and aliens coexist) have thrived and created nations whose strength rivals that of the CS. In Bolivia, freed Human and D-Bees formed the Megaversal Legion: a mercenary company with one of the highest levels of technology on Rifts Earth. ==== Europe ==== England has become a vast wilderness again, broken up by the occasional giant Millennium Tree or feudal kingdom, complete with a New Camelot and a new King Arthur, partially being manipulated by an alien intelligence disguised as Merlin. Also the magic of
Model collapse
Model collapse, also known by other names such as "AI inbreeding", "AI cannibalism", "Habsburg AI", and "model autophagy disorder" or "MAD" is a phenomenon noted in artificial intelligence studies, where machine learning models gradually degrade due to errors coming from uncurated synthetic data, or due to training on the outputs of another model such as prior versions of itself. It is unclear to what extent the phenomenon threatens the long-term development of such models, and some techniques have been proposed to mitigate the effect. == Characteristics == Shumailov et al. coined the term to describe two specific stages to the degradation of machine learning models: early model collapse and late model collapse: In early model collapse, the model begins losing information about the tails of the distribution – mostly affecting minority data. Later work highlighted that early model collapse is hard to notice, since overall performance may appear to improve, while the model loses performance on minority data. In late model collapse, the model loses a significant proportion of its performance, confusing concepts and losing most of its variance. == Mechanism == Using synthetic data as training data can lead to issues with the quality and reliability of the trained model. Model collapse occurs for three main reasons: functional approximation errors sampling errors learning errors Importantly, it happens in even the simplest of models, where not all of the error sources are present. In more complex models the errors often compound, leading to faster collapse. == Disagreement over real-world impact == Some researchers and commentators on model collapse warn that the phenomenon could fundamentally threaten future generative AI development: As AI-generated data is shared on the Internet, it will inevitably end up in future training datasets, which are often crawled from the Internet. If training on "slop" (large quantities of unlabeled synthetic data) inevitably leads to model collapse, this could therefore pose a difficult problem. However, recently, other researchers have disagreed with this argument, showing that if synthetic data accumulates alongside human-generated data, model collapse is avoided. The researchers argue that data accumulating over time is a more realistic description of reality than deleting all existing data every year, and that the real-world impact of model collapse may not be as catastrophic as feared. An alternative branch of the literature investigates the use of machine learning detectors and watermarking to identify model generated data and filter it out. == Mathematical models of the phenomenon == === 1D Gaussian model === In 2024, a first attempt has been made at illustrating collapse for the simplest possible model — a single dimensional normal distribution fit using unbiased estimators of mean and variance, computed on samples from the previous generation. To make this more precise, we say that original data follows a normal distribution X 0 ∼ N ( μ , σ 2 ) {\displaystyle X^{0}\sim {\mathcal {N}}(\mu ,\sigma ^{2})} , and we possess M 0 {\displaystyle M_{0}} samples X j 0 {\displaystyle X_{j}^{0}} for j ∈ { 1 , … , M 0 } {\displaystyle j\in {\{\,1,\dots ,M_{0}\,{}\}}} . Denoting a general sample X j i {\displaystyle X_{j}^{i}} as sample j ∈ { 1 , … , M i } {\displaystyle j\in {\{\,1,\dots ,M_{i}\,{}\}}} at generation i {\displaystyle i} , then the next generation model is estimated using the sample mean and variance: μ i + 1 = 1 M i ∑ j X j i ; σ i + 1 2 = 1 M i − 1 ∑ j ( X j i − μ i + 1 ) 2 . {\displaystyle \mu _{i+1}={\frac {1}{M_{i}}}\sum _{j}X_{j}^{i};\quad \sigma _{i+1}^{2}={\frac {1}{M_{i}-1}}\sum _{j}(X_{j}^{i}-\mu _{i+1})^{2}.} Leading to a conditionally normal next generation model X j i + 1 | μ i + 1 , σ i + 1 ∼ N ( μ i + 1 , σ i + 1 2 ) {\displaystyle X_{j}^{i+1}|\mu _{i+1},\;\sigma _{i+1}\sim {\mathcal {N}}(\mu _{i+1},\sigma _{i+1}^{2})} . In theory, this is enough to calculate the full distribution of X j i {\displaystyle X_{j}^{i}} . However, even after the first generation, the full distribution is no longer normal: It follows a variance-gamma distribution. To continue the analysis, instead of writing the probability density function at each generation, it is possible to explicitly construct them in terms of independent random variables using Cochran's theorem. To be precise, μ 1 {\displaystyle \mu _{1}} and σ 1 {\displaystyle \sigma _{1}} are independent, with μ 1 ∼ N ( μ , σ 2 M 0 ) {\displaystyle \mu _{1}\sim {\mathcal {N}}\left(\mu ,{\frac {\sigma ^{2}}{M_{0}}}\right)} and ( M 0 − 1 ) σ 1 2 ∼ σ 2 Γ ( M 0 − 1 2 , 1 2 ) {\displaystyle (M_{0}-1)\,\sigma _{1}^{2}\sim \sigma ^{2}\,\Gamma \left({\frac {M_{0}-1}{2}},{\frac {1}{2}}\right)} , following a Gamma distribution. Denoting with Z {\displaystyle Z} Gaussian random variables distributed according to N ( 0 , 1 ) {\displaystyle {\mathcal {N}}(0,1)} and with S i {\displaystyle S^{i}} random variables distributed with 1 M i − 1 − 1 Γ ( M i − 1 − 1 2 , 1 2 ) {\displaystyle {\frac {1}{M_{i-1}-1}}\Gamma \left({\frac {M_{i-1}-1}{2}},{\frac {1}{2}}\right)} , it turns out to be possible to write samples at each generation as X j 0 = μ + σ Z j 0 , {\textstyle X_{j}^{0}=\mu +\sigma Z_{j}^{0},} X j 1 = μ + σ M 0 Z 1 + σ S 1 Z j 1 , {\textstyle X_{j}^{1}=\mu +{\frac {\sigma }{\sqrt {M_{0}}}}Z^{1}+\sigma {\sqrt {S^{1}}}Z_{j}^{1},} and more generally X j n = μ + σ M 0 Z 1 + σ M 1 S 1 Z 2 + ⋯ + σ M n − 1 S 1 × ⋯ × S n − 1 Z n + σ S 1 × ⋯ × S n Z j n . {\displaystyle X_{j}^{n}=\mu +{\frac {\sigma }{\sqrt {M_{0}}}}Z^{1}+{\frac {\sigma }{\sqrt {M_{1}}}}{\sqrt {S^{1}}}Z^{2}+\dots +{\frac {\sigma }{\sqrt {M_{n-1}}}}{\sqrt {S^{1}\times \dots \times S^{n-1}}}Z^{n}+\sigma {\sqrt {S^{1}\times \dots \times S^{n}}}Z_{j}^{n}.} Note, that these are not joint distributions, as Z n {\displaystyle Z^{n}} and S n {\displaystyle S^{n}} depend directly on Z j n − 1 {\displaystyle Z_{j}^{n-1}} , but when considering X j n {\displaystyle X_{j}^{n}} on its own the formula above provides all the information about the full distribution. To analyse the model collapse, we can first calculate variance and mean of samples at generation n {\displaystyle n} . This would tell us what kind of distributions we expect to arrive at after n {\displaystyle n} generations. It is possible to find its exact value in closed form, but the mean and variance of the square root of gamma distribution are expressed in terms of gamma functions, making the result quite clunky. Following, it is possible to expand all results to second order in each of 1 / M i {\displaystyle 1/M_{i}} , assuming each sample size to be large. It is then possible to show that 1 σ 2 Var ( X j n ) = 1 M 0 + 1 M 1 + ⋯ + 1 M n − 1 + 1 + O ( M i − 2 ) . {\displaystyle {\frac {1}{\sigma ^{2}}}\operatorname {Var} (X_{j}^{n})={\frac {1}{M_{0}}}+{\frac {1}{M_{1}}}+\dots +{\frac {1}{M_{n-1}}}+1+{\mathcal {O}}\left(M_{i}^{-2}\right).} And if all sample sizes M i = M {\displaystyle M_{i}=M} are constant, this diverges linearly as n → ∞ {\displaystyle n\to \infty } : Var ( X j n ) = σ 2 ( 1 + n M ) ; E ( X j n ) = μ . {\displaystyle \operatorname {Var} (X_{j}^{n})=\sigma ^{2}\left(1+{\frac {n}{M}}\right);\quad \mathbb {E} (X_{j}^{n})=\mu .} This is the same scaling as for a single dimensional Gaussian random walk. However, divergence of the variance of X j n {\displaystyle X_{j}^{n}} does not directly provide any information about the corresponding estimates of μ n + 1 {\displaystyle \mu _{n+1}} and σ n + 1 {\displaystyle \sigma _{n+1}} , particularly how different they are from the original μ {\displaystyle \mu } and σ {\displaystyle \sigma } . It turns out to be possible to calculate the distance between the true distribution and the approximated distribution at step n + 1 {\displaystyle n+1} , using the Wasserstein-2 distance (which is also sometimes referred to as risk): E [ W 2 2 ( N ( μ , σ 2 ) , N ( μ n + 1 , σ n + 1 2 ) ) ] = 3 2 σ 2 ( 1 M 0 + 1 M 1 + ⋯ + 1 M n ) + O ( M i − 2 ) , {\displaystyle \mathbb {E} \left[\mathbb {W} _{2}^{2}\left({\mathcal {N}}(\mu ,\sigma ^{2}),{\mathcal {N}}(\mu _{n+1},\sigma _{n+1}^{2})\right)\right]={\frac {3}{2}}\sigma ^{2}\left({\frac {1}{M_{0}}}+{\frac {1}{M_{1}}}+\dots +{\frac {1}{M_{n}}}\right)+{\mathcal {O}}\left(M_{i}^{-2}\right),} Var [ W 2 2 ( N ( μ , σ 2 ) , N ( μ n + 1 , σ n + 1 2 ) ) ] = 1 2 σ 4 ( 3 M 0 2 + 3 M 1 2 + ⋯ + 3 M n 2 + ∑ i ≠ j 4 M i M j ) + O ( M i − 3 ) . {\displaystyle \operatorname {Var} \left[\mathbb {W} _{2}^{2}\left({\mathcal {N}}(\mu ,\sigma ^{2}),{\mathcal {N}}(\mu _{n+1},\sigma _{n+1}^{2})\right)\right]={\frac {1}{2}}\sigma ^{4}\left({\frac {3}{M_{0}^{2}}}+{\frac {3}{M_{1}^{2}}}+\dots +{\frac {3}{M_{n}^{2}}}+\sum _{i\neq j}{\frac {4}{M_{i}M_{j}}}\right)+{\mathcal {O}}\left(M_{i}^{-3}\right).} This directly shows why model collapse occurs in this simple model. Due to errors from re-sampling the approximated distribution, each generation ends up corresponding to a
Deep learning in photoacoustic imaging
Photoacoustic imaging (PA) is based on the photoacoustic effect, in which optical absorption causes a rise in temperature, which causes a subsequent rise in pressure via thermo-elastic expansion. This pressure rise propagates through the tissue and is sensed via ultrasonic transducers. Due to the proportionality between the optical absorption, the rise in temperature, and the rise in pressure, the ultrasound pressure wave signal can be used to quantify the original optical energy deposition within the tissue. Photoacoustic imaging has applications of deep learning in both photoacoustic computed tomography (PACT) and photoacoustic microscopy (PAM). PACT utilizes wide-field optical excitation and an array of unfocused ultrasound transducers. Similar to other computed tomography methods, the sample is imaged at multiple view angles, which are then used to perform an inverse reconstruction algorithm based on the detection geometry (typically through universal backprojection, modified delay-and-sum, or time reversal ) to elicit the initial pressure distribution within the tissue. PAM on the other hand uses focused ultrasound detection combined with weakly focused optical excitation (acoustic resolution PAM or AR-PAM) or tightly focused optical excitation (optical resolution PAM or OR-PAM). PAM typically captures images point-by-point via a mechanical raster scanning pattern. At each scanned point, the acoustic time-of-flight provides axial resolution while the acoustic focusing yields lateral resolution. == Applications of deep learning in PACT == The first application of deep learning in PACT was by Reiter et al. in which a deep neural network was trained to learn spatial impulse responses and locate photoacoustic point sources. The resulting mean axial and lateral point location errors on 2,412 of their randomly selected test images were 0.28 mm and 0.37 mm respectively. After this initial implementation, the applications of deep learning in PACT have branched out primarily into removing artifacts from acoustic reflections, sparse sampling, limited-view, and limited-bandwidth. There has also been some recent work in PACT toward using deep learning for wavefront localization. There have been networks based on fusion of information from two different reconstructions to improve the reconstruction using deep learning fusion based networks. === Using deep learning to locate photoacoustic point sources === Traditional photoacoustic beamforming techniques modeled photoacoustic wave propagation by using detector array geometry and the time-of-flight to account for differences in the PA signal arrival time. However, this technique failed to account for reverberant acoustic signals caused by acoustic reflection, resulting in acoustic reflection artifacts that corrupt the true photoacoustic point source location information. In Reiter et al., a convolutional neural network (similar to a simple VGG-16 style architecture) was used that took pre-beamformed photoacoustic data as input and outputted a classification result specifying the 2-D point source location. ==== Deep learning for PA wavefront localization ==== Johnstonbaugh et al. was able to localize the source of photoacoustic wavefronts with a deep neural network. The network used was an encoder-decoder style convolutional neural network. The encoder-decoder network was made of residual convolution, upsampling, and high field-of-view convolution modules. A Nyquist convolution layer and differentiable spatial-to-numerical transform layer were also used within the architecture. Simulated PA wavefronts served as the input for training the model. To create the wavefronts, the forward simulation of light propagation was done with the NIRFast toolbox and the light-diffusion approximation, while the forward simulation of sound propagation was done with the K-Wave toolbox. The simulated wavefronts were subjected to different scattering mediums and Gaussian noise. The output for the network was an artifact free heat map of the targets axial and lateral position. The network had a mean error rate of less than 30 microns when localizing target below 40 mm and had a mean error rate of 1.06 mm for localizing targets between 40 mm and 60 mm. With a slight modification to the network, the model was able to accommodate multi target localization. A validation experiment was performed in which pencil lead was submerged into an intralipid solution at a depth of 32 mm. The network was able to localize the lead's position when the solution had a reduced scattering coefficient of 0, 5, 10, and 15 cm−1. The results of the network show improvements over standard delay-and-sum or frequency-domain beamforming algorithms and Johnstonbaugh proposes that this technology could be used for optical wavefront shaping, circulating melanoma cell detection, and real-time vascular surgeries. === Removing acoustic reflection artifacts (in the presence of multiple sources and channel noise) === Building on the work of Reiter et al., Allman et al. utilized a full VGG-16 architecture to locate point sources and remove reflection artifacts within raw photoacoustic channel data (in the presence of multiple sources and channel noise). This utilization of deep learning trained on simulated data produced in the MATLAB k-wave library, and then later reaffirmed their results on experimental data. === Ill-posed PACT reconstruction === In PACT, tomographic reconstruction is performed, in which the projections from multiple solid angles are combined to form an image. When reconstruction methods like filtered backprojection or time reversal, are ill-posed inverse problems due to sampling under the Nyquist-Shannon's sampling requirement or with limited-bandwidth/view, the resulting reconstruction contains image artifacts. Traditionally these artifacts were removed with slow iterative methods like total variation minimization, but the advent of deep learning approaches has opened a new avenue that utilizes a priori knowledge from network training to remove artifacts. In the deep learning methods that seek to remove these sparse sampling, limited-bandwidth, and limited-view artifacts, the typical workflow involves first performing the ill-posed reconstruction technique to transform the pre-beamformed data into a 2-D representation of the initial pressure distribution that contains artifacts. Then, a convolutional neural network (CNN) is trained to remove the artifacts, in order to produce an artifact-free representation of the ground truth initial pressure distribution. ==== Using deep learning to remove sparse sampling artifacts ==== When the density of uniform tomographic view angles is under what is prescribed by the Nyquist-Shannon's sampling theorem, it is said that the imaging system is performing sparse sampling. Sparse sampling typically occurs as a way of keeping production costs low and improving image acquisition speed. The typical network architectures used to remove these sparse sampling artifacts are U-net and Fully Dense (FD) U-net. Both of these architectures contain a compression and decompression phase. The compression phase learns to compress the image to a latent representation that lacks the imaging artifacts and other details. The decompression phase then combines with information passed by the residual connections in order to add back image details without adding in the details associated with the artifacts. FD U-net modifies the original U-net architecture by including dense blocks that allow layers to utilize information learned by previous layers within the dense block. Another technique was proposed using a simple CNN based architecture for removal of artifacts and improving the k-wave image reconstruction. ==== Removing limited-view artifacts with deep learning ==== When a region of partial solid angles are not captured, generally due to geometric limitations, the image acquisition is said to have limited-view. As illustrated by the experiments of Davoudi et al., limited-view corruptions can be directly observed as missing information in the frequency domain of the reconstructed image. Limited-view, similar to sparse sampling, makes the initial reconstruction algorithm ill-posed. Prior to deep learning, the limited-view problem was addressed with complex hardware such as acoustic deflectors and full ring-shaped transducer arrays, as well as solutions like compressed sensing, weighted factor, and iterative filtered backprojection. The result of this ill-posed reconstruction is imaging artifacts that can be removed by CNNs. The deep learning algorithms used to remove limited-view artifacts include U-net and FD U-net, as well as generative adversarial networks (GANs) and volumetric versions of U-net. One GAN implementation of note improved upon U-net by using U-net as a generator and VGG as a discriminator, with the Wasserstein metric and gradient penalty to stabilize training (WGAN-GP). ==== Pixel-wise interpolation
Constructive cooperative coevolution
The constructive cooperative coevolutionary algorithm (also called C3) is a global optimisation algorithm in artificial intelligence based on the multi-start architecture of the greedy randomized adaptive search procedure (GRASP). It incorporates the existing cooperative coevolutionary algorithm (CC). The considered problem is decomposed into subproblems. These subproblems are optimised separately while exchanging information in order to solve the complete problem. An optimisation algorithm, usually but not necessarily an evolutionary algorithm, is embedded in C3 for optimising those subproblems. The nature of the embedded optimisation algorithm determines whether C3's behaviour is deterministic or stochastic. The C3 optimisation algorithm was originally designed for simulation-based optimisation but it can be used for global optimisation problems in general. Its strength over other optimisation algorithms, specifically cooperative coevolution, is that it is better able to handle non-separable optimisation problems. An improved version was proposed later, called the Improved Constructive Cooperative Coevolutionary Differential Evolution (C3iDE), which removes several limitations with the previous version. A novel element of C3iDE is the advanced initialisation of the subpopulations. C3iDE initially optimises the subpopulations in a partially co-adaptive fashion. During the initial optimisation of a subpopulation, only a subset of the other subcomponents is considered for the co-adaptation. This subset increases stepwise until all subcomponents are considered. This makes C3iDE very effective on large-scale global optimisation problems (up to 1000 dimensions) compared to cooperative coevolutionary algorithm (CC) and Differential evolution. The improved algorithm has then been adapted for multi-objective optimization. == Algorithm == As shown in the pseudo code below, an iteration of C3 exists of two phases. In Phase I, the constructive phase, a feasible solution for the entire problem is constructed in a stepwise manner. Considering a different subproblem in each step. After the final step, all subproblems are considered and a solution for the complete problem has been constructed. This constructed solution is then used as the initial solution in Phase II, the local improvement phase. The CC algorithm is employed to further optimise the constructed solution. A cycle of Phase II includes optimising the subproblems separately while keeping the parameters of the other subproblems fixed to a central blackboard solution. When this is done for each subproblem, the found solution are combined during a "collaboration" step, and the best one among the produced combinations becomes the blackboard solution for the next cycle. In the next cycle, the same is repeated. Phase II, and thereby the current iteration, are terminated when the search of the CC algorithm stagnates and no significantly better solutions are being found. Then, the next iteration is started. At the start of the next iteration, a new feasible solution is constructed, utilising solutions that were found during the Phase I of the previous iteration(s). This constructed solution is then used as the initial solution in Phase II in the same way as in the first iteration. This is repeated until one of the termination criteria for the optimisation is reached, e.g. a maximum number of evaluations. {Sphase1} ← ∅ while termination criteria not satisfied do if {Sphase1} = ∅ then {Sphase1} ← SubOpt(∅, 1) end if while pphase1 not completely constructed do pphase1 ← GetBest({Sphase1}) {Sphase1} ← SubOpt(pphase1, inext subproblem) end while pphase2 ← GetBest({Sphase1}) while not stagnate do {Sphase2} ← ∅ for each subproblem i do {Sphase2} ← SubOpt(pphase2,i) end for {Sphase2} ← Collab({Sphase2}) pphase2 ← GetBest({Sphase2}) end while end while == Multi-objective optimisation == The multi-objective version of the C3 algorithm is a Pareto-based algorithm which uses the same divide-and-conquer strategy as the single-objective C3 optimisation algorithm . The algorithm again starts with the advanced constructive initial optimisations of the subpopulations, considering an increasing subset of subproblems. The subset increases until the entire set of all subproblems is included. During these initial optimisations, the subpopulation of the latest included subproblem is evolved by a multi-objective evolutionary algorithm. For the fitness calculations of the members of the subpopulation, they are combined with a collaborator solution from each of the previously optimised subpopulations. Once all subproblems' subpopulations have been initially optimised, the multi-objective C3 optimisation algorithm continues to optimise each subproblem in a round-robin fashion, but now collaborator solutions from all other subproblems' subspopulations are combined with the member of the subpopulation that is being evaluated. The collaborator solution is selected randomly from the solutions that make up the Pareto-optimal front of the subpopulation. The fitness assignment to the collaborator solutions is done in an optimistic fashion (i.e. an "old" fitness value is replaced when the new one is better). == Applications == The constructive cooperative coevolution algorithm has been applied to different types of problems, e.g. a set of standard benchmark functions, optimisation of sheet metal press lines and interacting production stations. The C3 algorithm has been embedded with, amongst others, the differential evolution algorithm and the particle swarm optimiser for the subproblem optimisations.