Currently Reading.....longer papers that I'm working through...
No this isn't the normal link roundup....
I have these papers open in tabs and as I work through them, I thought I’d go ahead and share them out here.
AnyGPT: Unified Multimodal LLM with Discrete Sequence Modeling: “We introduce AnyGPT, an any-to-any multimodal language model that utilizes discrete representations for the unified processing of various modalities, including speech, text, images, and music. AnyGPT can be trained stably without any alterations to the current large language model (LLM) architecture or training paradigms. Instead, it relies exclusively on data-level preprocessing, facilitating the seamless integration of new modalities into LLMs, akin to the incorporation of new languages.”
Forty years in the making: A systematic review of the megatrends literature: “The current paper presents the first systematic review of megatrends publications from 1982 to 2022, including a bibliometric analysis of publishing trends. From an analysis of 267 studies, we present a novel diagnostic tool for defining megatrends that brings together the consensus characteristics of megatrends (e.g., its multidisciplinary nature, drivers of change, breadth, long time span and forward-focused lens). Our findings also reveal validity and reliability issues associated with megatrend studies, and from this, we present a standardised approach for developing and validating megatrends that improves upon these shortcomings.”
Prompting Diverse Ideas: Increasing AI Idea Variance: In this context, we find that (1) pools of ideas generated by GPT-4 with various plausible prompts are less diverse than ideas generated by groups of human subjects (2) the diversity of AI generated ideas can be substantially improved using prompt engineering (3) Chain-of-Thought (CoT) prompting leads to the highest diversity of ideas of all prompts we evaluated and was able to come close to what is achieved by groups of human subjects. It also was capable of generating the highest number of unique ideas of any prompt we studied.
This AI Paper Unveils a New Method for Statistically-Guaranteed Text Generation Using Non-Exchangeable Conformal Prediction: “To address this, the study draws on advancements in nearest-neighbor language modeling and machine translation. They propose dynamically generating calibration sets during inference to uphold statistical guarantees. Before delving into the method, it’s essential to grasp two key concepts: Conformal Prediction, known for its statistical coverage guarantees, and Non-exchangeable Conformal Prediction, which tackles the miscalibration caused by distributional drifts in non-i.i.d. scenarios by assigning relevance-weighted calibration data points.”
Generative Artificial Intelligence and Evaluating Strategic Decisions: “Strategic decisions involve uncertainty and are often irreversible. Hence, predicting the value of strategic alternatives is important for decision making. We investigate the potential role of generative AI in evaluating strategic alternatives. Using a sample of 60 business models, we examine the extent to which business model rankings made by large language models (LLMs) agree with those of human experts.”
AI-Augmented Predictions: LLM Assistants Improve Human Forecasting Accuracy: “Large language models (LLMs) show impressive capabilities, matching and sometimes exceeding human performance in many domains. This study explores the potential of LLMs to augment judgement in forecasting tasks. We evaluated the impact on forecasting accuracy of two GPT-4-Turbo assistants: one designed to provide high-quality advice ('superforecasting'), and the other designed to be overconfident and base-rate-neglecting. Participants (N = 991) had the option to consult their assigned LLM assistant throughout the study, in contrast to a control group that used a less advanced model (DaVinci-003) without direct forecasting support.”
Automated Social Science: Language Models as Scientist and Subjects: “We present an approach for automatically generating and testing, in silico, social scientific hypotheses. This automation is made possible by recent advances in large language models (LLM), but the key feature of the approachis the use of structural causal models. Structural causal models provide a language to state hypotheses, a blueprint for constructing LLM-based agents, an experimental design, and a plan for data analysis. The fitted structural causal model becomes an object available for prediction or the planning of follow-on experiments. We demonstrate the approach with several scenarios: a negotiation, a bail hearing, a job interview, and an auction. In each case, causal relationships are both proposed and tested by the system, finding evidence for some and not others. We provide evidence that the insights from these simulations of social interactions are not available to the LLM purely through direct elicitation. When given its proposed structural causal model for each scenario, the LLM is good at predicting the signs of estimated effects, but it cannot reliably predict the magnitudes of those estimates. In the auction experiment, the in silico simulation results closely match the predictions of auction theory, but elicited predictions of the clearing prices from the LLM are inaccurate. However, the LLM’s predictions are dramatically improved if the model can condition on the fitted structural causal model. In short, the LLM knows more than it can (immediately) tell.”
Empowering Biomedical Discovery with AI Agents: “We envision 'AI scientists' as systems capable of skeptical learning and reasoning that empower biomedical research through collaborative agents that integrate machine learning tools with experimental platforms. Rather than taking humans out of the discovery process, biomedical AI agents combine human creativity and expertise with AI's ability to analyze large datasets, navigate hypothesis spaces, and execute repetitive tasks. AI agents are proficient in a variety of tasks, including self-assessment and planning of discovery workflows. These agents use large language models and generative models to feature structured memory for continual learning and use machine learning tools to incorporate scientific knowledge, biological principles, and theories. AI agents can impact areas ranging from hybrid cell simulation, programmable control of phenotypes, and the design of cellular circuits to the development of new therapies.”
FLOW MATCHING FOR GENERATIVE MODELING: “We introduce a new paradigm for generative modeling built on Continuous Normalizing Flows (CNFs), allowing us to train CNFs at unprecedented scale. Specifically, we present the notion of Flow Matching (FM), a simulation-free approach for training CNFs based on regressing vector fields of fixed conditional probability paths. Flow Matching is compatible with a general family of Gaussian probability paths for transforming between noise and data samples—which subsumes existing diffusion paths as specific instances. Interestingly, we find that employing FM with diffusion paths results in a more robust and stable alternative for training diffusion models. Furthermore, Flow Matching opens the door to training CNFs with other, non-diffusion probability paths.”
Scalable Diffusion Models with Transformers: “We explore a new class of diffusion models based on the transformer architecture. We train latent diffusion models of images, replacing the commonly-used U-Net backbone with a transformer that operates on latent patches. We analyze the scalability of our Diffusion Transformers (DiTs) through the lens of forward pass complexity as measured by Gflops. We find that DiTs with higher Gflops -- through increased transformer depth/width or increased number of input tokens -- consistently have lower FID.”
Cultural evolution in populations of Large Language Models: “Research in cultural evolution aims at providing causal explanations for the change of culture over time. Over the past decades, this field has generated an important body of knowledge, using experimental, historical, and computational methods. While computational models have been very successful at generating testable hypotheses about the effects of several factors, such as population structure or transmission biases, some phenomena have so far been more complex to capture using agent-based and formal models. This is in particular the case for the effect of the transformations of social information induced by evolved cognitive mechanisms. We here propose that leveraging the capacity of Large Language Models (LLMs) to mimic human behavior may be fruitful to address this gap. On top of being an useful approximation of human cultural dynamics, multi-agents models featuring generative agents are also important to study for their own sake. Indeed, as artificial agents are bound to participate more and more to the evolution of culture, it is crucial to better understand the dynamics of machine-generated cultural evolution. We here present a framework for simulating cultural evolution in populations of LLMs, allowing the manipulation of variables known to be important in cultural evolution, such as network structure, personality, and the way social information is aggregated and transformed. The software we developed for conducting these simulations is open-source and features an intuitive user-interface, which we hope will help to build bridges between the fields of cultural evolution and generative artificial intelligence.”
Playing with Metaphors. Connecting Experiential Futures and Critical Futures Studies: “This paper explores working with metaphors to link Experiential Futures (XF) and Critical Futures Studies (CFS). We introduce the Systematic Metaphor Analysis (SMA) developed by Schmitt and his colleagues to reconstruct metaphors that emerge during the creation of XF and to make them explicit for critical reflection. We illustrate this work by applying SMA on metaphors of AI that emerged during an XF, namely a role-playing game (RPG). The introduction of SMA enriches Futures Studies’ critical approaches. Furthermore, our contribution to playing and working with metaphors hopefully intensifies and inspires further discussions on the connections between XF and CFS.”
What is the price of a skill? The value of complementarity: “As skills are seldomly applied in isolation, we propose that complementarity strongly determines a skill's economic value. For 962 skills, we demonstrate that their value is determined by complementarity – that is, how many different skills, ideally of high value, a competency can be combined with. We show that the value of a skill is relative, as it depends on the skill background of the worker. For most skills, their value is highest when used in combination with skills of a different type.”
The Curiouser Nature of Trends: A Process Thesis of Sociocultural Trend Developments in Iterations of Mindsets and Practices: “This essay aims to address the development of sociocultural trends within the scope of Trend Studies. It puts forward a thesis for a conceptual model that lays new foundations to understand the development of trends in terms of the articulations between mindsets and associated representations, practices and artifacts that are made tangible in the visible world. The model takes an initial inspiration from semiology, focusing on cultural information, meaning and idea transfer to underline that changes are always in flux. This contribution can be applied to better understand a trend’s nature, based on its current iteration and considering former mutations, and in a futures exercise to extrapolate short and medium-term evolutions or reactions to its nature.”
The Digital Turn in Business Anthropology: “This essay examines the emerging “digital turn” in business anthropology, a phenomenon propelled by the increasing prevalence and influence of digital technologies. Despite the significant underrepresentation of digital anthropology in current literature within the Journal of Business Anthropology, its relevance to the traditionally focused areas of organizational culture, marketing, consumer research, advertising, and user experience is irrefutable, given the rapid digitalization of the business landscape.”
Collingridge and the dilemma of control: Towards responsible and accountable innovation: “The paper critically reviews the work of David Collingridge in the light of contemporary concerns about responsibility and accountability in innovation, public engagement with science and technology, and the role of scientific expertise in technology policy. Given continued interest in his thoughts on the ‘social control of technology’, and the ‘dilemma of control’, this attention is both timely and overdue.”
Automated Social Science: Language Models as Scientist and Subjects: “We present an approach for automatically generating and testing, in silico, social scientific hypotheses. This automation is made possible by recent advances in large language models (LLM), but the key feature of the approach is the use of structural causal models. Structural causal models provide a language to state hypotheses, a blueprint for constructing LLM-based agents, an experimental design, and a plan for data analysis. The fitted structural causal model becomes an object available for prediction or the planning of follow-on experiments.”