Search

We study recent research advances that improve large language models through efficient pre-training and scaling, and open datasets and tools. We combine these advances to introduce Cerebras-GPT, a family of open compute-optimal language models scaled from 111M to 13B parameters. We train Cerebras-GPT models on the Eleuther Pile dataset following DeepMind Chinchilla scaling rules for efficient pre-training (highest accuracy for a given compute budget). We characterize the predictable power-law scaling and compare Cerebras-GPT with other publicly-available models to show all Cerebras-GPT models have state-of-the-art training efficiency on both pre-training and downstream objectives. We describe our learnings including how Maximal Update Parameterization ($\mu$P) can further improve large model scaling, improving accuracy and hyperparameter predictability at scale. We release our pre-trained models and code, making this paper the first open and reproducible work comparing compute-optimal model scaling to models trained on fixed dataset sizes. Cerebras-GPT models are available on HuggingFace: https://huggingface.co/cerebras., Comment: 13 pages main text, 16 pages appendix, 13 figures

We introduce the Bittensor Language Model, called "BTLM-3B-8K", a new state-of-the-art 3 billion parameter open-source language model. BTLM-3B-8K was trained on 627B tokens from the SlimPajama dataset with a mixture of 2,048 and 8,192 context lengths. BTLM-3B-8K outperforms all existing 3B parameter models by 2-5.5% across downstream tasks. BTLM-3B-8K is even competitive with some 7B parameter models. Additionally, BTLM-3B-8K provides excellent long context performance, outperforming MPT-7B-8K and XGen-7B-8K on tasks up to 8,192 context length. We trained the model on a cleaned and deduplicated SlimPajama dataset; aggressively tuned the \textmu P hyperparameters and schedule; used ALiBi position embeddings; and adopted the SwiGLU nonlinearity. On Hugging Face, the most popular models have 7B parameters, indicating that users prefer the quality-size ratio of 7B models. Compacting the 7B parameter model to one with 3B parameters, with little performance impact, is an important milestone. BTLM-3B-8K needs only 3GB of memory with 4-bit precision and takes 2.5x less inference compute than 7B models, helping to open up access to a powerful language model on mobile and edge devices. BTLM-3B-8K is available under an Apache 2.0 license on Hugging Face: https://huggingface.co/cerebras/btlm-3b-8k-base.

The study aims to find an appropriate model to extract insights from the sales of a Pharmaceutical Distribution Company (PDC) and make it available in an interactive and readable manner for the company. In PDCs, it is highly important to obtain a good approximation of the medicine needs, due to the short shelf life of many medicines and the need to control stock levels. The presented method is a combination of analysis and interactive visualization tools along with prediction. In this paper, we explore the use of Support Vector Regression algorithm for the sales prediction of individual products. The proposed model helps to present the sales data in a better way such that understanding the trends and seasonality becomes easier for the PDCs. The dataset has information of hourly, daily, weekly and monthly sales of the drugs and hence the end results also give us a likely classified understanding of the sales. The study of the results obtained, suggest that the proposed model may be considered appropriate for product sales prediction.

The study aims to find an appropriate model to extract insights from the sales of a Pharmaceutical Distribution Company (PDC) and make it available in an interactive and readable manner for the company. In PDCs, it is highly important to obtain a good approximation of the medicine needs, due to the short shelf life of many medicines and the need to control stock levels. The presented method is a combination of analysis and interactive visualization tools along with prediction. In this paper, we explore the use of Support Vector Regression algorithm for the sales prediction of individual products. The proposed model helps to present the sales data in a better way such that understanding the trends and seasonality becomes easier for the PDCs. The dataset has information of hourly, daily, weekly and monthly sales of the drugs and hence the end results also give us a likely classified understanding of the sales. The study of the results obtained, suggest that the proposed model may be considered appropriate for product sales prediction.

The study aims to find an appropriate model to extract insights from the sales of a Pharmaceutical Distribution Company (PDC) and make it available in an interactive and readable manner for the company. In PDCs, it is highly important to obtain a good approximation of the medicine needs, due to the short shelf life of many medicines and the need to control stock levels. The presented method is a combination of analysis and interactive visualization tools along with prediction. In this paper, we explore the use of Support Vector Regression algorithm for the sales prediction of individual products. The proposed model helps to present the sales data in a better way such that understanding the trends and seasonality becomes easier for the PDCs. The dataset has information of hourly, daily, weekly and monthly sales of the drugs and hence the end results also give us a likely classified understanding of the sales. The study of the results obtained, suggest that the proposed model may be considered appropriate for product sales prediction.

It is well established that resource quantity and elemental stoichiometry play major roles in shaping below and aboveground plant biodiversity, but their importance for shaping microbial diversity in soil remains unclear. Here, we used statistical modeling on a regional database covering 179 locations and six ecosystem types across Scotland to evaluate the roles of total carbon (C), nitrogen (N) and phosphorus (P) availabilities and ratios, together with land use, climate and biotic and abiotic factors, in determining regional scale patterns of soil bacterial diversity. We found that bacterial diversity and composition were primarily driven by variation in soil resource stoichiometry (total C:N:P ratios), itself linked to different land uses, and secondarily driven by other important biodiversity drivers such as climate, soil spatial heterogeneity, soil pH, root influence (plant-soil microbe interactions) and microbial biomass (soil microbe-microbe interactions). In aggregate, these findings provide evidence that nutrient stoichiometry is a strong predictor of bacterial diversity and composition at a regional scale. This article is protected by copyright. All rights reserved.

Soil stores more carbon (C) than plants and atmosphere combined and it is vulnerable to increased microbial respiration under projected global changes including land-use change and future climate scenarios (mainly elevated temperature). Land-use change is known to have a direct impact on soil organic C and soil respiration (Rs) but the mechanisms that drive these changes remain debatable. Similarly, recent studies and simulation models predict that Rs will respond positively to projected climate warming. However, there are significant uncertainties in the magnitude and mechanisms of this feedback response of Rs to global change. To identify the mechanisms of Rs response to land-use change and climate warming, we first investigated Rs from different land use types. Soil respiration was estimated seasonally from four different Scottish land uses: moorland, birch woodland, grassland and pine forest (n = 24). Our results demonstrated that despite a dramatic loss of total C and nitrogen (N) in the soils under birch trees, the Rs in the birch woodland was similar to that of the moorland and pine forest, with Rs in the grassland being significantly higher. The microbial community structure, estimated by Multiplex Terminal-Restriction Fragment Length Polymorphism (MT-RFLP) and 454 pyrosequencing, was significantly different under each land use type. A strong correlation of Rs with soil properties (pH, inorganic N, C:N ratio and moisture) and with microbial community structure was identified.To test the impact of elevated temperature on Rs and to identify potential microbial mechanisms, we performed laboratory incubation studies. Soils from different land uses were incubated at 7 °C (mean annual temperature (MAT) in Scotland) and 10 °C (MAT + 3 °C) with and without the presence of a labile (13C-glucose) and recalcitrant (13C-lignin) form of C to identify the active groups of microbes and to determine the role of substrate avai

IMPORTANCE Schizophrenia (SCZ) is a devastating psychiatric condition. Identifying the specific genetic variants and pathways that increase susceptibility to SCZ is critical to improve disease understanding and address the urgent need for new drug targets. OBJECTIVE To identify SCZ susceptibility genes. DESIGN We integrated results from a meta-analysis of 18 genome-wide association studies (GWAS) involving 1 085 772 single-nucleotide polymorphisms (SNPs) and 6 databases that showed significant informativeness for SCZ. The 9380 most promising SNPs were then specifically genotyped in an independent family-based replication study that, after quality control, consisted of 8107 SNPs. SETTING Linkage meta-analysis, brain transcriptome meta-analysis, candidate gene database, OMIM, relevant mouse studies, and expression quantitative trait locus databases. PATIENTS We included 11 185 cases and 10 768 control subjects from 6 databases and, after quality control 6298 individuals (including 3286 cases) from 1811 nuclear families. MAIN OUTCOMES AND MEASURES Case-control status for SCZ. RESULTS Replication results showed a highly significant enrichment of SNPs with small P values. Of the SNPs with replication values of P <. 01, the proportion of SNPs that had the same direction of effects as in the GWAS meta-analysis was 89% in the combined ancestry group (sign test, P < 2.20 × 10-16) and 93% in subjects of European ancestry only (P < 2.20 × 10-16). Our results supported the major histocompatibility complex region showing a 3.7-fold overall enrichment of replication values of P < .01 in subjects from European ancestry. We replicated SNPs in TCF4 (P = 2.53 × 10-10) and NOTCH4 (P = 3.16 × 10-7) that are among the most robust SCZ findings. More novel findings included POM121L2 (P = 3.51 × 10-7), AS3MT (P = 9.01 × 10-7), CNNM2 (P = 6.07 × 10-7), and NT5C2 (P = 4.09 × 10-7). To explore the many small effects, we performed pathway analyses. The most significant pathways inv

IMPORTANCE Schizophrenia (SCZ) is a devastating psychiatric condition. Identifying the specific genetic variants and pathways that increase susceptibility to SCZ is critical to improve disease understanding and address the urgent need for new drug targets. OBJECTIVE To identify SCZ susceptibility genes. DESIGN We integrated results from a meta-analysis of 18 genome-wide association studies (GWAS) involving 1 085 772 single-nucleotide polymorphisms (SNPs) and 6 databases that showed significant informativeness for SCZ. The 9380 most promising SNPs were then specifically genotyped in an independent family-based replication study that, after quality control, consisted of 8107 SNPs. SETTING Linkage meta-analysis, brain transcriptome meta-analysis, candidate gene database, OMIM, relevant mouse studies, and expression quantitative trait locus databases. PATIENTS We included 11 185 cases and 10 768 control subjects from 6 databases and, after quality control 6298 individuals (including 3286 cases) from 1811 nuclear families. MAIN OUTCOMES AND MEASURES Case-control status for SCZ. RESULTS Replication results showed a highly significant enrichment of SNPs with small P values. Of the SNPs with replication values of P <. 01, the proportion of SNPs that had the same direction of effects as in the GWAS meta-analysis was 89% in the combined ancestry group (sign test, P < 2.20 × 10-16) and 93% in subjects of European ancestry only (P < 2.20 × 10-16). Our results supported the major histocompatibility complex region showing a 3.7-fold overall enrichment of replication values of P < .01 in subjects from European ancestry. We replicated SNPs in TCF4 (P = 2.53 × 10-10) and NOTCH4 (P = 3.16 × 10-7) that are among the most robust SCZ findings. More novel findings included POM121L2 (P = 3.51 × 10-7), AS3MT (P = 9.01 × 10-7), CNNM2 (P = 6.07 × 10-7), and NT5C2 (P = 4.09 × 10-7). To explore the many small effects, we performed pathway analyses. The most significant pathways inv

Performance appraisal (PA) is a management concept that is meant to improve individual and, concomitantly, organisational performance. Although performance appraisal systems have come under a plethora of criticism, they continue to be a platform of human resource management and development systems. Performance appraisals are utilised in an attempt to measure employee performance in organisations. Oftentimes the outcomes of these appraisals affect the employee’s retention, promotion, or salary. An effective performance appraisal system should improve the morale, motivation, and overall productivity of an organisation by identifying employees’ strengths and addressing areas that need improvement. This research study presents an investigation in as far as the evaluation of the staff appraisal practices at the Lesotho College of Education (LCE) are concerned. The methodology employed in this investigation includes a literature study on human resource management and development; performance management; and performance appraisal in the context of higher education. An opinion survey among academic staff was undertaken by means of semi-structured interviews and a quantitative closed-ended questionnaire with enhancement by means of qualitative, open-ended questions. The results of the survey were finally interpreted in terms of four research questions pertaining to the problem being investigated. Despite the strengths of the staff appraisal practices identified at the LCE, there are also critical weaknesses in the appraisal system such as poor communication between supervisors and subordinates. The researcher therefore concluded that the staff appraisal practices at the LCE are undoubtedly problematic in nature and can, at best, only be partly effective. Finally, a number of recommendations were formulated under the following five headings: Design, Policy Formulation and Documentation of the system; Dissemination of the system; Implementation of the system; Staff Development

Report prepared as part of the coursework required for REL 167, Speaking Stones., Description and interpretation of the monument to Laura Knapp (d. 1882, 3 years old).

Report prepared as part of the coursework required for REL 167, Speaking Stones., Description and interpretation of the monument to Laura Knapp (d. 1882, 3 years old).