Search

Your search keyword '"Ma, Chi A"' showing total 3,419 results
Start Over Author "Ma, Chi A"
3,419 results on '"Ma, Chi A"'

1. First Activations Matter: Training-Free Methods for Dynamic Activation in Large Language Models

Author

Ma, Chi, Huang, Mincong, Zhang, Ying, Wang, Chao, Wang, Yujie, Yu, Lei, Liu, Chuan, and Lin, Wei

Subjects
Computer Science - Computation and Language, Computer Science - Machine Learning
Abstract

Dynamic activation (DA) techniques, such as DejaVu and MoEfication, have demonstrated their potential to significantly enhance the inference efficiency of large language models (LLMs). However, these techniques often rely on ReLU activation functions or require additional parameters and training to maintain performance. This paper introduces a training-free Threshold-based Dynamic Activation(TDA) method that leverage sequence information to exploit the inherent sparsity of models across various architectures. This method is designed to accelerate generation speed by 18-25\% without significantly compromising task performance, thereby addressing the limitations of existing DA techniques. Moreover, we delve into the root causes of LLM sparsity and theoretically analyze two of its critical features: history-related activation uncertainty and semantic-irrelevant activation inertia. Our comprehensive analyses not only provide a robust theoretical foundation for DA methods but also offer valuable insights to guide future research in optimizing LLMs for greater efficiency and effectiveness.

Published
2024

2. The differences in the origination and properties of the near-Earth solar wind between solar cycles 23 and 24

Author

Shi, Xinzheng, Fu, Hui, Huang, Zhenghua, Yan, Limei, Ma, Chi, Huangfu, Chenxi, Song, Hongqiang, and Xia, Lidong

Subjects
Astrophysics - Solar and Stellar Astrophysics, Physics - Space Physics
Abstract

The dependence of the sources and properties of the near-Earth solar wind on solar cycle activity is an important issue in solar and space physics. We use the improved two-step mapping procedure that takes into account the initial acceleration processes to trace the near-Earth solar winds back to their source regions from 1999 to 2020, covering solar cycles (SCs) 23 and 24. Then the solar wind is categorized into coronal hole (CH), active region (AR), and quiet Sun (QS) solar wind based on the source region types. We find that the proportions of CH and AR (QS) wind during SC 23 are higher (lower) than those during SC 24. During solar maximum and declining phases, the magnetic field strength, speed, helium abundance (AHe), and charge states of all three types of solar wind during SC 23 are generally higher than those during SC 24. During solar minimum, these parameters of solar wind are generally lower during SC 23 than those during SC 24. There is a significant decrease in the charge states of all three types of solar wind during the solar minimum of SC 23. The present statistical results demonstrate that the sources and properties of the solar wind are both influenced by solar cycle amplitude. The temperatures of AR, QS, and CH regions exhibit significant difference at low altitudes, whereas they are almost uniform at high altitudes., Comment: 21 pages, 11 figures

Published
2024

3. MOYU: A Theoretical Study on Massive Over-activation Yielded Uplifts in LLMs

Author

Ma, Chi, Huang, Mincong, Wang, Chao, Wang, Yujie, and Yu, Lei

Subjects
Computer Science - Machine Learning
Abstract

Massive Over-activation Yielded Uplifts(MOYU) is an inherent property of large language models, and dynamic activation(DA) based on the MOYU property is a clever yet under-explored strategy designed to accelerate inference in these models. Existing methods that utilize MOYU often face a significant 'Impossible Trinity': struggling to simultaneously maintain model performance, enhance inference speed, and extend applicability across various architectures. Due to the theoretical ambiguities surrounding MOYU, this paper elucidates the root cause of the MOYU property and outlines the mechanisms behind two primary limitations encountered by current DA methods: 1) history-related activation uncertainty, and 2) semantic-irrelevant activation inertia. Our analysis not only underscores the limitations of current dynamic activation strategies within large-scale LLaMA models but also proposes opportunities for refining the design of future sparsity schemes.

Published
2024

4. Dynamic Activation Pitfalls in LLaMA Models: An Empirical Study

Author

Ma, Chi, Huang, Mincong, Wang, Chao, Wang, Yujie, and Yu, Lei

Subjects
Computer Science - Machine Learning
Abstract

In this work, we systematically investigate the efficacy of dynamic activation mechanisms within the LLaMA family of language models. Despite the potential of dynamic activation methods to reduce computation and increase speed in models using the ReLU activation function, our empirical findings have uncovered several inherent pitfalls in the current dynamic activation schemes. Through extensive experiments across various dynamic activation strategies, we demonstrate that LLaMA models usually underperform when compared to their ReLU counterparts, particularly in scenarios demanding high sparsity ratio. We attribute these deficiencies to a combination of factors: 1) the inherent complexity of dynamically predicting activation heads and neurons; 2) the inadequate sparsity resulting from activation functions; 3) the insufficient preservation of information resulting from KV cache skipping. Our analysis not only sheds light on the limitations of dynamic activation in the context of large-scale LLaMA models but also proposes roadmaps for enhancing the design of future sparsity schemes.

Published
2024

7. Frame Optimization in Speech Emotion Recognition Based on Improved EMD and SVM Algorithms

Author

Guo, Chuan-Jie, Jin, Shu-Ya, Zhang, Yu-Zhe, Ma, Chi-Yuan, Adeel, Muhammad, Tao, Zhi-Yong, Akan, Ozgur, Editorial Board Member, Bellavista, Paolo, Editorial Board Member, Cao, Jiannong, Editorial Board Member, Coulson, Geoffrey, Editorial Board Member, Dressler, Falko, Editorial Board Member, Ferrari, Domenico, Editorial Board Member, Gerla, Mario, Editorial Board Member, Kobayashi, Hisashi, Editorial Board Member, Palazzo, Sergio, Editorial Board Member, Sahni, Sartaj, Editorial Board Member, Shen, Xuemin, Editorial Board Member, Stan, Mircea, Editorial Board Member, Jia, Xiaohua, Editorial Board Member, Zomaya, Albert Y., Editorial Board Member, and Wang, Junyi, editor

Published
2025
Full Text
View/download PDF

8. Innate protection against intrarectal SIV acquisition by a live SHIV vaccine.

Author

Sui, Yongjun, Meyer, Thomas, Fennessey, Christine, Keele, Brandon, Dadkhah, Kimia, Ma, Chi, LaBranche, Celia, Breed, Matthew, Kramer, Josh, Li, Jianping, Howe, Savannah, Ferrari, Guido, Williams, LaTonya, Cam, Maggie, Kelly, Michael, Shen, Xiaoying, Tomaras, Georgia, Montefiori, David, Greten, Tim, Miller, Chris, and Berzofsky, Jay

Subjects
AIDS vaccine, AIDS/HIV, Innate immunity, Vaccines, Animals, Simian Acquired Immunodeficiency Syndrome, Simian Immunodeficiency Virus, SAIDS Vaccines, Macaca mulatta, Immunity, Innate, CD8-Positive T-Lymphocytes, Antibodies, Viral, Male, Vaccines, Attenuated
Abstract

Identifying immune correlates of protection is a major challenge in AIDS vaccine development. Anti-Envelope antibodies have been considered critical for protection against SIV/HIV (SHIV) acquisition. Here, we evaluated the efficacy of an SHIV vaccine against SIVmac251 challenge, where the role of antibody was excluded, as there was no cross-reactivity between SIV and SHIV envelope antibodies. After 8 low-dose intrarectal challenges with SIVmac251, 12 SHIV-vaccinated animals demonstrated efficacy, compared with 6 naive controls, suggesting protection was achieved in the absence of anti-envelope antibodies. Interestingly, CD8+ T cells (and some NK cells) were not essential for preventing viral acquisition, as none of the CD8-depleted macaques were infected by SIVmac251 challenges. Initial investigation of protective innate immunity revealed that protected animals had elevated pathways related to platelet aggregation/activation and reduced pathways related to interferon and responses to virus. Moreover, higher expression of platelet factor 4 on circulating platelet-leukocyte aggregates was associated with reduced viral acquisition. Our data highlighted the importance of innate immunity, identified mechanisms, and may provide opportunities for novel HIV vaccines or therapeutic strategy development.

Published
2024

11. Re-evaluating the Memory-balanced Pipeline Parallelism: BPipe

Author

Huang, Mincong, Wang, Chao, Ma, Chi, Zhang, Yineng, Zhang, Peng, and Yu, Lei

Subjects
Computer Science - Machine Learning, Computer Science - Computation and Language, Computer Science - Distributed, Parallel, and Cluster Computing
Abstract

Pipeline parallelism is an essential technique in the training of large-scale Transformer models. However, it suffers from imbalanced memory consumption, leading to insufficient memory utilization. The BPipe technique was proposed to address this issue and has proven effective in the GPT-3 model. Nevertheless, our experiments have not yielded similar benefits for LLaMA training. Additionally, BPipe only yields negligible benefits for GPT-3 training when applying flash attention. We analyze the underlying causes of the divergent performance of BPipe on GPT-3 and LLaMA. Furthermore, we introduce a novel method to estimate the performance of BPipe.

Published
2024

19. Variant STAT4 and Response to Ruxolitinib in an Autoinflammatory Syndrome

Author

Baghdassarian, Hratch, Blackstone, Sarah A, Clay, Owen S, Philips, Rachael, Matthiasardottir, Brynja, Nehrebecky, Michele, Hua, Vivian K, McVicar, Rachael, Liu, Yang, Tucker, Suzanne M, Randazzo, Davide, Deuitch, Natalie, Rosenzweig, Sofia, Mark, Adam, Sasik, Roman, Fisch, Kathleen M, Pimpale Chavan, Pallavi, Eren, Elif, Watts, Norman R, Ma, Chi A, Gadina, Massimo, Schwartz, Daniella M, Sanyal, Anwesha, Werner, Giffin, Murdock, David R, Horita, Nobuyuki, Chowdhury, Shimul, Dimmock, David, Jepsen, Kristen, Remmers, Elaine F, Goldbach-Mansky, Raphaela, Gahl, William A, O'Shea, John J, Milner, Joshua D, Lewis, Nathan E, Chang, Johanna, Kastner, Daniel L, Torok, Kathryn, Oda, Hirotsugu, Putnam, Christopher D, and Broderick, Lori

Subjects
Biomedical and Clinical Sciences, Clinical Sciences, Genetics, Human Genome, 2.1 Biological and endogenous factors, Inflammatory and immune system, Janus Kinases, Nitriles, Pyrazoles, Pyrimidines, Scleroderma, Systemic, Autoimmune Diseases, Mutation, Missense, Gain of Function Mutation, Dermatologic Agents, Anti-Inflammatory Agents, Medical and Health Sciences, General & Internal Medicine, Biomedical and clinical sciences, Health sciences
Abstract

BackgroundDisabling pansclerotic morphea (DPM) is a rare systemic inflammatory disorder, characterized by poor wound healing, fibrosis, cytopenias, hypogammaglobulinemia, and squamous-cell carcinoma. The cause is unknown, and mortality is high.MethodsWe evaluated four patients from three unrelated families with an autosomal dominant pattern of inheritance of DPM. Genomic sequencing independently identified three heterozygous variants in a specific region of the gene that encodes signal transducer and activator of transcription 4 (STAT4). Primary skin fibroblast and cell-line assays were used to define the functional nature of the genetic defect. We also assayed gene expression using single-cell RNA sequencing of peripheral-blood mononuclear cells to identify inflammatory pathways that may be affected in DPM and that may respond to therapy.ResultsGenome sequencing revealed three novel heterozygous missense gain-of-function variants in STAT4. In vitro, primary skin fibroblasts showed enhanced interleukin-6 secretion, with impaired wound healing, contraction of the collagen matrix, and matrix secretion. Inhibition of Janus kinase (JAK)-STAT signaling with ruxolitinib led to improvement in the hyperinflammatory fibroblast phenotype in vitro and resolution of inflammatory markers and clinical symptoms in treated patients, without adverse effects. Single-cell RNA sequencing revealed expression patterns consistent with an immunodysregulatory phenotype that were appropriately modified through JAK inhibition.ConclusionsGain-of-function variants in STAT4 caused DPM in the families that we studied. The JAK inhibitor ruxolitinib attenuated the dermatologic and inflammatory phenotype in vitro and in the affected family members. (Funded by the American Academy of Allergy, Asthma, and Immunology Foundation and others.).

Published
2023

25. Task Scheduling for Low-Orbit Satellite Edge Computing Based on Deep Q Network

Author

Biao, Xiaowei, Zhang, Mingming, Gou, Baowei, Ma, Chi, Ye, Nan, Chinese Institute of Command and Control, Angrisani, Leopoldo, Series Editor, Arteaga, Marco, Series Editor, Chakraborty, Samarjit, Series Editor, Chen, Shanben, Series Editor, Chen, Tan Kay, Series Editor, Dillmann, Rüdiger, Series Editor, Duan, Haibin, Series Editor, Ferrari, Gianluigi, Series Editor, Ferre, Manuel, Series Editor, Jabbari, Faryar, Series Editor, Jia, Limin, Series Editor, Kacprzyk, Janusz, Series Editor, Khamis, Alaa, Series Editor, Kroeger, Torsten, Series Editor, Li, Yong, Series Editor, Liang, Qilian, Series Editor, Martín, Ferran, Series Editor, Ming, Tan Cher, Series Editor, Minker, Wolfgang, Series Editor, Misra, Pradeep, Series Editor, Mukhopadhyay, Subhas, Series Editor, Ning, Cun-Zheng, Series Editor, Nishida, Toyoaki, Series Editor, Oneto, Luca, Series Editor, Panigrahi, Bijaya Ketan, Series Editor, Pascucci, Federica, Series Editor, Qin, Yong, Series Editor, Seng, Gan Woon, Series Editor, Speidel, Joachim, Series Editor, Veiga, Germano, Series Editor, Wu, Haitao, Series Editor, Zamboni, Walter, Series Editor, and Tan, Kay Chen, Series Editor

Published
2024
Full Text
View/download PDF

27. Full Speed Range Field Oriented Control Method for PMSM

Author

Liu, Donghui, Zhao, Leiting, Kan, Liu, Li, Liran, Ma, Chi, Yue, Zheng, Jin, Zheming, Diao, Lijun, Angrisani, Leopoldo, Series Editor, Arteaga, Marco, Series Editor, Chakraborty, Samarjit, Series Editor, Chen, Jiming, Series Editor, Chen, Shanben, Series Editor, Chen, Tan Kay, Series Editor, Dillmann, Rüdiger, Series Editor, Duan, Haibin, Series Editor, Ferrari, Gianluigi, Series Editor, Ferre, Manuel, Series Editor, Jabbari, Faryar, Series Editor, Jia, Limin, Series Editor, Kacprzyk, Janusz, Series Editor, Khamis, Alaa, Series Editor, Kroeger, Torsten, Series Editor, Li, Yong, Series Editor, Liang, Qilian, Series Editor, Martín, Ferran, Series Editor, Ming, Tan Cher, Series Editor, Minker, Wolfgang, Series Editor, Misra, Pradeep, Series Editor, Mukhopadhyay, Subhas, Series Editor, Ning, Cun-Zheng, Series Editor, Nishida, Toyoaki, Series Editor, Oneto, Luca, Series Editor, Panigrahi, Bijaya Ketan, Series Editor, Pascucci, Federica, Series Editor, Qin, Yong, Series Editor, Seng, Gan Woon, Series Editor, Speidel, Joachim, Series Editor, Veiga, Germano, Series Editor, Wu, Haitao, Series Editor, Zamboni, Walter, Series Editor, Zhang, Junjie James, Series Editor, Tan, Kay Chen, Series Editor, Yang, Jianwei, editor, Liu, Zhigang, editor, Diao, Lijun, editor, and An, Min, editor

Published
2024
Full Text
View/download PDF

28. Establishment of Power Cable Dynamic Three-Dimensional Thermal Model Serving Digital Twin

Author

Zhang, Boyu, Ma, Chi, Li, Jiangtao, Angrisani, Leopoldo, Series Editor, Arteaga, Marco, Series Editor, Chakraborty, Samarjit, Series Editor, Chen, Jiming, Series Editor, Chen, Shanben, Series Editor, Chen, Tan Kay, Series Editor, Dillmann, Rüdiger, Series Editor, Duan, Haibin, Series Editor, Ferrari, Gianluigi, Series Editor, Ferre, Manuel, Series Editor, Jabbari, Faryar, Series Editor, Jia, Limin, Series Editor, Kacprzyk, Janusz, Series Editor, Khamis, Alaa, Series Editor, Kroeger, Torsten, Series Editor, Li, Yong, Series Editor, Liang, Qilian, Series Editor, Martín, Ferran, Series Editor, Ming, Tan Cher, Series Editor, Minker, Wolfgang, Series Editor, Misra, Pradeep, Series Editor, Mukhopadhyay, Subhas, Series Editor, Ning, Cun-Zheng, Series Editor, Nishida, Toyoaki, Series Editor, Oneto, Luca, Series Editor, Panigrahi, Bijaya Ketan, Series Editor, Pascucci, Federica, Series Editor, Qin, Yong, Series Editor, Seng, Gan Woon, Series Editor, Speidel, Joachim, Series Editor, Veiga, Germano, Series Editor, Wu, Haitao, Series Editor, Zamboni, Walter, Series Editor, Zhang, Junjie James, Series Editor, Tan, Kay Chen, Series Editor, Dong, Xuzhu, editor, and Cai, Li, editor

Published
2024
Full Text
View/download PDF

29. Light Accumulation Map for Natural Foliage Scene Generation

Author

Shen, Ruien, Ma, Chi Weng, Dong, Deli, Xiao, Shuangjiu, Goos, Gerhard, Founding Editor, Hartmanis, Juris, Founding Editor, Bertino, Elisa, Editorial Board Member, Gao, Wen, Editorial Board Member, Steffen, Bernhard, Editorial Board Member, Yung, Moti, Editorial Board Member, Sheng, Bin, editor, Bi, Lei, editor, Kim, Jinman, editor, Magnenat-Thalmann, Nadia, editor, and Thalmann, Daniel, editor

Published
2024
Full Text
View/download PDF

30. Stress Intensity Factor Solution for Fractured Steel W-Shaped Beams Considering Non-overlapping Cracking Behavior

Author

Ma, Chi Hieu, Lam, Van Phong, Nguyen, Danh Thao, Le, Tuan Anh, Van Hong Bui, Linh, Do, Bach, di Prisco, Marco, Series Editor, Chen, Sheng-Hong, Series Editor, Vayas, Ioannis, Series Editor, Kumar Shukla, Sanjay, Series Editor, Sharma, Anuj, Series Editor, Kumar, Nagesh, Series Editor, Wang, Chien Ming, Series Editor, Cui, Zhen-Dong, Series Editor, Reddy, J. N., editor, Luong, Van Hai, editor, and Le, Anh Tuan, editor

Published
2024
Full Text
View/download PDF

46. Clinically relevant small-molecule promotes nerve repair and visual function recovery.

Author

Au, Ngan, Kumar, Gajendra, Asthana, Pallavi, Gao, Fuying, Kawaguchi, Riki, Chang, Raymond, So, Kwok, Hu, Yang, Geschwind, Daniel, Coppola, Giovanni, and Ma, Chi

Abstract

Adult mammalian injured axons regenerate over short-distance in the peripheral nervous system (PNS) while the axons in the central nervous system (CNS) are unable to regrow after injury. Here, we demonstrated that Lycium barbarum polysaccharides (LBP), purified from Wolfberry, accelerated long-distance axon regeneration after severe peripheral nerve injury (PNI) and optic nerve crush (ONC). LBP not only promoted intrinsic growth capacity of injured neurons and function recovery after severe PNI, but also induced robust retinal ganglion cell (RGC) survival and axon regeneration after ONC. By using LBP gene expression profile signatures to query a Connectivity map database, we identified a Food and Drug Administration (FDA)-approved small-molecule glycopyrrolate, which promoted PNS axon regeneration, RGC survival and sustained CNS axon regeneration, increased neural firing in the superior colliculus, and enhanced visual target re-innervations by regenerating RGC axons leading to a partial restoration of visual function after ONC. Our study provides insights into repurposing of FDA-approved small molecule for nerve repair and function recovery.

Published
2022

Catalog

Books, media, physical & digital resources
See catalog results