Search

Your search keyword '"Huang, Jiani"' showing total 242 results
Start Over Author "Huang, Jiani"
242 results on '"Huang, Jiani"'

1. Learning Geospatial Region Embedding with Heterogeneous Graph

Author

Zou, Xingchen, Huang, Jiani, Hao, Xixuan, Yang, Yuhao, Wen, Haomin, Yan, Yibo, Huang, Chao, and Liang, Yuxuan

Subjects
Computer Science - Machine Learning, Computer Science - Artificial Intelligence
Abstract

Learning effective geospatial embeddings is crucial for a series of geospatial applications such as city analytics and earth monitoring. However, learning comprehensive region representations presents two significant challenges: first, the deficiency of effective intra-region feature representation; and second, the difficulty of learning from intricate inter-region dependencies. In this paper, we present GeoHG, an effective heterogeneous graph structure for learning comprehensive region embeddings for various downstream tasks. Specifically, we tailor satellite image representation learning through geo-entity segmentation and point-of-interest (POI) integration for expressive intra-regional features. Furthermore, GeoHG unifies informative spatial interdependencies and socio-environmental attributes into a powerful heterogeneous graph to encourage explicit modeling of higher-order inter-regional relationships. The intra-regional features and inter-regional correlations are seamlessly integrated by a model-agnostic graph learning framework for diverse downstream tasks. Extensive experiments demonstrate the effectiveness of GeoHG in geo-prediction tasks compared to existing methods, even under extreme data scarcity (with just 5% of training data). With interpretable region representations, GeoHG exhibits strong generalization capabilities across regions. We will release code and data upon paper notification.

Published
2024

2. Graph Machine Learning in the Era of Large Language Models (LLMs)

Author

Fan, Wenqi, Wang, Shijie, Huang, Jiani, Chen, Zhikai, Song, Yu, Tang, Wenzhuo, Mao, Haitao, Liu, Hui, Liu, Xiaorui, Yin, Dawei, and Li, Qing

Subjects
Computer Science - Machine Learning, Computer Science - Artificial Intelligence, Computer Science - Computation and Language, Computer Science - Social and Information Networks
Abstract

Graphs play an important role in representing complex relationships in various domains like social networks, knowledge graphs, and molecular discovery. With the advent of deep learning, Graph Neural Networks (GNNs) have emerged as a cornerstone in Graph Machine Learning (Graph ML), facilitating the representation and processing of graph structures. Recently, LLMs have demonstrated unprecedented capabilities in language tasks and are widely adopted in a variety of applications such as computer vision and recommender systems. This remarkable success has also attracted interest in applying LLMs to the graph domain. Increasing efforts have been made to explore the potential of LLMs in advancing Graph ML's generalization, transferability, and few-shot learning ability. Meanwhile, graphs, especially knowledge graphs, are rich in reliable factual knowledge, which can be utilized to enhance the reasoning capabilities of LLMs and potentially alleviate their limitations such as hallucinations and the lack of explainability. Given the rapid progress of this research direction, a systematic review summarizing the latest advancements for Graph ML in the era of LLMs is necessary to provide an in-depth understanding to researchers and practitioners. Therefore, in this survey, we first review the recent developments in Graph ML. We then explore how LLMs can be utilized to enhance the quality of graph features, alleviate the reliance on labeled data, and address challenges such as graph heterogeneity and out-of-distribution (OOD) generalization. Afterward, we delve into how graphs can enhance LLMs, highlighting their abilities to enhance LLM pre-training and inference. Furthermore, we investigate various applications and discuss the potential future directions in this promising field.

Published
2024

4. Improved Logical Reasoning of Language Models via Differentiable Symbolic Programming

Author

Zhang, Hanlin, Huang, Jiani, Li, Ziyang, Naik, Mayur, and Xing, Eric

Subjects
Computer Science - Artificial Intelligence, Computer Science - Computation and Language, Computer Science - Machine Learning
Abstract

Pre-trained large language models (LMs) struggle to perform logical reasoning reliably despite advances in scale and compositionality. In this work, we tackle this challenge through the lens of symbolic programming. We propose DSR-LM, a Differentiable Symbolic Reasoning framework where pre-trained LMs govern the perception of factual knowledge, and a symbolic module performs deductive reasoning. In contrast to works that rely on hand-crafted logic rules, our differentiable symbolic reasoning framework efficiently learns weighted rules and applies semantic loss to further improve LMs. DSR-LM is scalable, interpretable, and allows easy integration of prior knowledge, thereby supporting extensive symbolic programming to robustly derive a logical conclusion. The results of our experiments suggest that DSR-LM improves the logical reasoning abilities of pre-trained language models, resulting in a significant increase in accuracy of over 20% on deductive reasoning benchmarks. Furthermore, DSR-LM outperforms a variety of competitive baselines when faced with systematic changes in sequence length., Comment: ACL 2023 Findings

Published
2023

5. LASER: A Neuro-Symbolic Framework for Learning Spatial-Temporal Scene Graphs with Weak Supervision

Author

Huang, Jiani, Li, Ziyang, Naik, Mayur, and Lim, Ser-Nam

Subjects
Computer Science - Computer Vision and Pattern Recognition, Computer Science - Machine Learning, Computer Science - Logic in Computer Science
Abstract

We propose LASER, a neuro-symbolic approach to learn semantic video representations that capture rich spatial and temporal properties in video data by leveraging high-level logic specifications. In particular, we formulate the problem in terms of alignment between raw videos and spatio-temporal logic specifications. The alignment algorithm leverages a differentiable symbolic reasoner and a combination of contrastive, temporal, and semantics losses. It effectively and efficiently trains low-level perception models to extract a fine-grained video representation in the form of a spatio-temporal scene graph that conforms to the desired high-level specification. To practically reduce the manual effort of obtaining ground truth labels, we derive logic specifications from captions by employing a large language model with a generic prompting template. In doing so, we explore a novel methodology that weakly supervises the learning of spatio-temporal scene graphs with widely accessible video-caption data. We evaluate our method on three datasets with rich spatial and temporal specifications: 20BN-Something-Something, MUGEN, and OpenPVSG. We demonstrate that our method learns better fine-grained video semantics than existing baselines.

Published
2023

6. Scallop: A Language for Neurosymbolic Programming

Author

Li, Ziyang, Huang, Jiani, and Naik, Mayur

Subjects
Computer Science - Programming Languages, Computer Science - Artificial Intelligence, Computer Science - Machine Learning
Abstract

We present Scallop, a language which combines the benefits of deep learning and logical reasoning. Scallop enables users to write a wide range of neurosymbolic applications and train them in a data- and compute-efficient manner. It achieves these goals through three key features: 1) a flexible symbolic representation that is based on the relational data model; 2) a declarative logic programming language that is based on Datalog and supports recursion, aggregation, and negation; and 3) a framework for automatic and efficient differentiable reasoning that is based on the theory of provenance semirings. We evaluate Scallop on a suite of eight neurosymbolic applications from the literature. Our evaluation demonstrates that Scallop is capable of expressing algorithmic reasoning in diverse and challenging AI tasks, provides a succinct interface for machine learning programmers to integrate logical domain knowledge, and yields solutions that are comparable or superior to state-of-the-art models in terms of accuracy. Furthermore, Scallop's solutions outperform these models in aspects such as runtime and data efficiency, interpretability, and generalizability.

Published
2023

8. Crowd-sourced machine learning prediction of long COVID using data from the National COVID Cohort Collaborative

Author

Aggarwal, Ataes, Agor, Joseph, Al-Amery, Amera, Aminu, Oluwatobiloba, Anand, Adit, Antonescu, Corneliu, Arora, Mehak, Asaduzzaman, Sayed, Asmussen, Tanner, Baghbanzadeh, Mahdi, Baker, Frazier, Bangert, Bridget, Bekhet, Laila, Bhattacharya, Biplab, Blase, Jenny, Butzin-Dozier, Zachary, Caffo, Brian, Chang, Hao, Chen, Zeyuan, Chen, Jiandong, Chiang, Jeffrey, Cho, Peter, Cockrell, Robert, Combs, Parker, Coyle, Jeremy, Crosby, Ciara, Dai, Zongyu, Dai, Ran, Danesharasteh, Anseh, Yildirim, Elif, Delong, Grant, Demilt, Ryan, Deng, Kaiwen, Dey, Sanjoy, Dhamdhere, Rohan, Dickson, Andrew, Dijour, Phoebe, Dinh, Dong, Dixon, Richard, Domi, Albi, Dutta, Souradeep, Elizondo, Mirna, Ertem, Zeynep, Feuerwerker, Solomon, Fliss, Danica, Fowler, Jennifer, Fu, Sunyang, Gardner, Kelly, Getty, Neil, Ghalwash, Mohamed, Gloster, Logan, Greer, Phil, Guan, Yuanfang, Ham, Colby, Hanoudi, Samer, Harper, Jeremy, Hendrix, Nathaniel, Hershkovich, Leeor, Hightower, Jake, Hu, Junjie, Huang, Jiani, Huang, Yu, Huang, Tongtong, Hur, Junguk, Isgut, Monica, Ismail, Hamid, Izmirlian, Grant, Jang, Kuk, Jemiyo, Christianah, Jeong, Hayoung, Ji, Yunwen, Ji, Xiayan, Jiang, Ming, Jiang, Sihang, Jiang, Xiaoqian, Jiang, Yuye, Johnson, Akin, Analyst, Zach, Kapse, Saarthak, Kartoun, Uri, KC, Dukka, Fard, Zahra, Kosfeld, Tim, Krichevsky, Spencer, Kuo, Mike, Larie, Dale, Lederer, Lauren, Leng, Shan, Li, Ziyang, Li, Hongyang, Li, Haodong, Li, Jianfu, Li, Tiantian, Liang, Xinwen, Liang, Hengyue, Liu, Feifan, Liu, Daniel, Luo, Gang, Munia, Tamanna, Madduri, Ravi, Madhira, Vithal, Mani, Shivali, Mansourifard, Farzaneh, Matson, Robert, Mertens, Andrew, Metsis, Vangelis, Meyer, Pablo, Mikhailova, Catherine, Miller, Dante, Milo, Christopher, Mitchell, Elliot, Modanwal, Gourav, Moore, Ronald, Morgenthaler, David, Musal, Rasim, Naik, Mayur, Nalawade, Vinit, Narain, Rohan, Narendrula, Saideep, Obiri, Alena, Okawa, Satoshi, Okechukwu, Chima, Olorunnisola, Toluwanimi, Ossowski, Tim, Parekh, Harsh, Park, Jean, Patel, Saaya, Patterson, Jason, Paul, Chetan, Peng, Le, Perkins, Diana, Pokharel, Suresh, Poplavskiy, Dmytro, Pryor, Zach, Pungitore, Sarah, Qin, Hong, Rababa, Salahaldeen, Rahman, Mahbubur, Rahmani, Elior, Rahnavard, Gholamali, Raihan, Md, Rajendran, Suraj, Ravichandran, Sarangan, Reddy, Chandan, Reyes, Abel, Roghanizad, Ali, Rouffa, Sean, Ruan, Xiaoyang, Saha, Arpita, Sawant, Sahil, Schiaffino, Melody, Seira, Diego, Sengupta, Saurav, Shalaev, Ruslan, Shetty, Gaurav, Shi, Seraphina, Shinguyen, Linh, Singh, Karnika, Sinha, Soumya, Socia, Damien, Stalians, Halen, Stavropoulos, Charalambos, Strube, Jan, Subramanian, Devika, Sun, Jiehuan, Sun, Ju, Sun, Chengkun, Sundararajan, Prathic, Talebi, Salmonn, Tawiah, Edward, Tesic, Jelena, Thiess, Mikaela, Tian, Raymond, Torre-Healy; Ming-Tse Tsai, Luke, Tyus, David, Vardhan, Madhurima, Velingker, Neelay, Walzer, Benjamin, Walzer, Jacob, Wang, Junda, Wang, Lu, Wang, Will, Wang, Jonathan, Wang, Yisen, Weatherly, Chad, Wu, Fanyou, Wu, Yifeng, Wu, Yinjun, Xia, Fangfang, Yan, Hao, Yang, Zhichao, Ye, Biao, Yin, Rui, Yin, Changyu, Yoo, Yun, You, Albert, Yu, June, Zanaj, Martin, Zaiman, Zachary, Zhang, Kai, Zhang, Xiaoyi, Zhang, Tianmai, Zhao, Zixuan, Zhi, Degui, Zhong, Yishan, Zhou, Huixue, Zhou, Andrea, Zhu, Yuanda, Zhu, Yitan, Zhu, Sophie, Adams, Meredith, Alexander, Caleb, Amor, Benjamin, Anzalone, Alfred, Bates, Benjamin, Beasley, Will, Bennett, Tellen, Bissell, Mark, Boudreau, Eilis, Bozzette, Samuel, Bradwell, Katie, Bramante, Carolyn, Brown, Don, Burgoon, Penny, Buse, John, Callahan, Tiffany, Cato, Kenrick, Chapman, Scott, Chute, Christopher, Clark, Jaylyn, Clark, Marshall, Cooper, Will, Cottrell, Lesley, Crowley, Karen, Deacy, Mariam, Dillon, Christopher, Eichmann, David, Emmett, Mary, Erwin-Cohen, Rebecca, Francis, Patricia, French, Evan, Fuentes, Rafael, Gabriel, Davera, Gagnier, Joel, Garbarini, Nicole, Ge, Jin, Gersing, Kenneth, Girvin, Andrew, Gordon, Valery, Graves, Alexis, Guinney, Justin, Haendel, Melissa, Hayanga, J.W., Hendricks, Brian, Hernandez, Wenndy, Hill, Elaine, Hillegass, William, Hong, Stephanie, Housman, Dan, Hurley, Robert, Islam, Jessica, Jawa, Randeep, Johnson, Steve, Kamaleswaran, Rishi, Kibbe, Warren, Koraishy, Farrukh, Kostka, Kristin, Kurilla, Michael, Lee, Adam, Lehmann, Harold, Liu, Hongfang, Loomba, Johanna, Madlock-Brown; Sandeep Mallipattu, Charisse, Manna, Amin, Mariona, Federico, Marti, Emily, Martin, Greg, Mathew, Jomol, Mazzotti, Diego, McMurry, Julie, Mehta, Hemalkumar, Michael, Sam, Miller, Robert, Misquitta, Leonie, Moffitt, Richard, Morris, Michele, Murray, Kimberly, Northington, Lavance, O’Neil, Shawn, Olex, Amy, Palchuk, Matvey, Patel, Brijesh, Patel, Rena, Payne, Philip, Pfaff, Emily, Pincavitch, Jami, Portilla, Lili, Prior, Fred, Pyarajan, Saiju, Pyles, Lee, Qureshi, Nabeel, Robinson, Peter, Rutter, Joni, Sadan, Ofer, Safdar, Nasia, Saha, Amit, Saltz, Joel, Saltz, Mary, Schmitt, Clare, Setoguchi, Soko, Sharafeldin, Noha, Sharathkumar, Anjali, Sheikh, Usman, Sidky, Hythem, Sokos, George, Southerland, Andrew, Spratt, Heidi, Starren, Justin, Subbian, Vignesh, Suver, Christine, Takemoto, Cliff, Temple-O'Connor, Meredith, Topaloglu, Umit, Vedula, Satyanarayana, Walden, Anita, Walters, Kellie, Ward-Caviness, Cavin, Wilcox, Adam, Wilkins, Ken, Williams, Andrew, Wu, Chunlei, Zampino, Elizabeth, Zhang, Xiaohan, Zhu, Richard, Bergquist, Timothy, Tariq, Adbul, Philips, Rachael V., Pirracchio, Romain, van der Laan, Mark, Colford, John M., Jr., Hubbard, Alan, Gao, Jifan, Chen, Guanhua, Stein, Adam, Long, Qi, Holmes, John, Mowery, Danielle, Wong, Eric, Parekh, Ravi, Getzen, Emily, and Blase, Jennifer

Published
2024
Full Text
View/download PDF

10. Relationship between idolatry and perceived social support in secondary school students： the mediating role of peer attachment and life satisfaction

Author

Li Jinlu, Shan Feipeng, Huang Jiani, and Huang Lieyu

Subjects
secondary school students, idolatry, peer attachment, life satisfaction, perceived social support, Psychology, BF1-990, Psychiatry, RC435-571
Abstract

BackgroundNowadays， idolatry is an important part of the spiritual life of secondary school students， making a significant impact on their physical and mental development. Previous research has examined the two-by-two relationship between idolatry， peer attachment， life satisfaction and perceived social support， but the potential mediating mechanisms between these variables remain to be explored.ObjectiveTo explore the mediating role of peer attachment and life satisfaction between idolatry and perceived social support， so as to provide references for mental health education for secondary school students.MethodsIn June 2022， 1 059 students currently enrolled in secondary school in Qiannan State， Guizhou Province were selected according to stratified random sampling method. In this study， assessment was performed using Celebrity Attitude Scale （CAS）， Peer Attachment Subscale from Revised Inventory of Parent and Peer Attachment （IPPA-R）， Satisfaction with Life Scale （SWLS） and Perceived Social Support Scale （PSSS）. Pearson correlation analysis was used for correlation test. Model 6 in the SPSS Macro Process 3.3 was used to examine the mediating effect of peer attachment and life satisfaction between idolatry and perceived social support in secondary school students.Results①CAS score indicated positive correlation with scores of IPPA-R Peer Attachment Subscale， SWLS and PSSS （r=0.117， 0.097， 0.115， P

Published
2023
Full Text
View/download PDF

13. A Penetration Path Planning Algorithm for UAV Based on Laguerre Diagram

Author

Li, Dan, Chen, Xu, Ding, Panpan, Huang, Jiani, Goos, Gerhard, Founding Editor, Hartmanis, Juris, Founding Editor, Bertino, Elisa, Editorial Board Member, Gao, Wen, Editorial Board Member, Steffen, Bernhard, Editorial Board Member, Yung, Moti, Editorial Board Member, Li, Bohan, editor, Yue, Lin, editor, Tao, Chuanqi, editor, Han, Xuming, editor, Calvanese, Diego, editor, and Amagasa, Toshiyuki, editor

Published
2023
Full Text
View/download PDF

20. Linguistic-Driven Partial Semantic Relevance Learning for Skeleton-Based Action Recognition.

Author

Chen, Qixiu, Liu, Yingan, Huang, Peng, and Huang, Jiani

Subjects
LANGUAGE models, MOTION analysis, HAIRBRUSHES, LINGUISTIC models, TOOTHBRUSHES
Abstract

Skeleton-based action recognition, renowned for its computational efficiency and indifference to lighting variations, has become a focal point in the realm of motion analysis. However, most current methods typically only extract global skeleton features, overlooking the potential semantic relationships among various partial limb motions. For instance, the subtle differences between actions such as "brush teeth" and "brush hair" are mainly distinguished by specific elements. Although combining limb movements provides a more holistic representation of an action, relying solely on skeleton points proves inadequate for capturing these nuances. Therefore, integrating detailed linguistic descriptions into the learning process of skeleton features is essential. This motivates us to explore integrating fine-grained language descriptions into the learning process of skeleton features to capture more discriminative skeleton behavior representations. To this end, we introduce a new Linguistic-Driven Partial Semantic Relevance Learning framework (LPSR) in this work. While using state-of-the-art large language models to generate linguistic descriptions of local limb motions and further constrain the learning of local motions, we also aggregate global skeleton point representations and textual representations (which generated from an LLM) to obtain a more generalized cross-modal behavioral representation. On this basis, we propose a cyclic attentional interaction module to model the implicit correlations between partial limb motions. Numerous ablation experiments demonstrate the effectiveness of the method proposed in this paper, and our method also obtains state-of-the-art results. [ABSTRACT FROM AUTHOR]

Published
2024
Full Text
View/download PDF

24. Amyloid‐β but not tau accumulation is strongly associated with longitudinal cognitive decline.

Author

Wang, Wenwen, Huang, Jiani, Qian, Shuangjie, Zheng, Yi, Yu, Xinyue, Jiang, Tao, Ai, Ruixue, Hou, Jialong, Ma, Enzi, Cai, Jinlai, He, Haijun, Wang, XinShi, and Xie, Chenglong

Abstract

Objective: Alzheimer's disease (AD) pathology is featured by the extracellular accumulation of amyloid‐β (Aβ) plaques and intracellular tau neurofibrillary tangles in the brain. We studied whether Aβ and tau accumulation are independently associated with future cognitive decline in the AD continuum. Methods: Data were acquired from the Alzheimer's Disease Neuroimaging Initiative (ADNI) public database. A total of 1272 participants were selected based on the availability of Aβ‐PET and CSF tau at baseline and of those 777 participants with follow‐up visits. Results: We found that Aβ‐PET and CSF tau pathology were related to cognitive decline across the AD clinical spectrum, both as potential predictors for dementia progression. Among them, Aβ‐PET (A + T− subjects) is an independent reliable predictor of longitudinal cognitive decline in terms of ADAS‐13, ADNI‐MEM, and MMSE scores rather than tau pathology (A − T+ subjects), indicating tau accumulation is not closely correlated with future cognitive impairment without being driven by Aβ deposition. Of note, a high percentage of APOE ε4 carriers with Aβ pathology (A+) develop poor memory and learning capacity. Interestingly, this condition is not recurrence in terms of the ADNI‐MEM domain when adding APOE ε4 status. Finally, the levels of Aβ‐PET SUVR related to glucose hypometabolism more strongly in subjects with A + T− than A − T+ both happen at baseline and longitudinal changes. Conclusions: In conclusion, Aβ‐PET alone without tau pathology (A + T−) measure is an independent reliable predictor of longitudinal cognitive decline but may nonetheless forecast different status of dementia progression. However, tau accumulation alone without Aβ pathology background (A − T+) was not enough to be an independent predictor of cognitive worsening. [ABSTRACT FROM AUTHOR]

Published
2024
Full Text
View/download PDF

25. CdS@CS/PVA inverse opal films as photocatalyst.

Author

Huang, Jiani, Chu, Zhaoran, Li, Junling, Shao, Wenting, Tang, Wenwei, Ning, Xuanjun, and Chen, Cheng

Subjects
MATERIALS science, PHOTONIC band gap structures, OPALS, FACE centered cubic structure, ELECTROMAGNETIC wave reflection
Published
2024
Full Text
View/download PDF

30. OsACA9, an Autoinhibited Ca 2+ -ATPase, Synergically Regulates Disease Resistance and Leaf Senescence in Rice.

Author

Wang, Xinyu, Wang, Ziyao, Lu, Yiduo, Huang, Jiani, Hu, Zhuoer, Lou, Junlei, Fan, Xinyue, Gu, Zhimin, Liu, Pengcheng, Ma, Bojun, and Chen, Xifeng

Subjects
CALCIUM ions, RECEPTOR-like kinases, RICE diseases & pests, REACTIVE oxygen species, GRAIN yields, HYBRID rice, ARABIDOPSIS thaliana
Abstract

Calcium (Ca2+) is a versatile intracellular second messenger that regulates several signaling pathways involved in growth, development, stress tolerance, and immune response in plants. Autoinhibited Ca2+-ATPases (ACAs) play an important role in the regulation of cellular Ca2+ homeostasis. Here, we systematically analyzed the putative OsACA family members in rice, and according to the phylogenetic tree of OsACAs, OsACA9 was clustered into a separated branch in which its homologous gene in Arabidopsis thaliana was reported to be involved in defense response. When the OsACA9 gene was knocked out by CRISPR/Cas9, significant accumulation of reactive oxygen species (ROS) was detected in the mutant lines. Meanwhile, the OsACA9 knock out lines showed enhanced disease resistance to both rice bacterial blight (BB) and bacterial leaf streak (BLS). In addition, compared to the wild-type (WT), the mutant lines displayed an early leaf senescence phenotype, and the agronomy traits of their plant height, panicle length, and grain yield were significantly decreased. Transcriptome analysis by RNA-Seq showed that the differentially expressed genes (DEGs) between WT and the Osaca9 mutant were mainly enriched in basal immune pathways and antibacterial metabolite synthesis pathways. Among them, multiple genes related to rice disease resistance, receptor-like cytoplasmic kinases (RLCKs) and cell wall-associated kinases (WAKs) genes were upregulated. Our results suggest that the Ca2+-ATPase OsACA9 may trigger oxidative burst in response to various pathogens and synergically regulate disease resistance and leaf senescence in rice. [ABSTRACT FROM AUTHOR]

Published
2024
Full Text
View/download PDF

45. LASER: Neuro-Symbolic Learning of Semantic Video Representations

Author

Huang, Jiani, Li, Ziyang, Jacobs, David, Naik, Mayur, and Lim, Ser-Nam

Subjects
FOS: Computer and information sciences, Computer Science - Machine Learning, Computer Science - Logic in Computer Science, Computer Vision and Pattern Recognition (cs.CV), Computer Science - Computer Vision and Pattern Recognition, Machine Learning (cs.LG), Logic in Computer Science (cs.LO)
Abstract

Modern AI applications involving video, such as video-text alignment, video search, and video captioning, benefit from a fine-grained understanding of video semantics. Existing approaches for video understanding are either data-hungry and need low-level annotation, or are based on general embeddings that are uninterpretable and can miss important details. We propose LASER, a neuro-symbolic approach that learns semantic video representations by leveraging logic specifications that can capture rich spatial and temporal properties in video data. In particular, we formulate the problem in terms of alignment between raw videos and specifications. The alignment process efficiently trains low-level perception models to extract a fine-grained video representation that conforms to the desired high-level specification. Our pipeline can be trained end-to-end and can incorporate contrastive and semantic loss functions derived from specifications. We evaluate our method on two datasets with rich spatial and temporal specifications: 20BN-Something-Something and MUGEN. We demonstrate that our method not only learns fine-grained video semantics but also outperforms existing baselines on downstream tasks such as video retrieval.

Published
2023
Full Text
View/download PDF

48. Tumor-induced erythroid precursor-differentiated myeloid cells mediate immunosuppression and curtail anti-PD-1/PD-L1 treatment efficacy

Author

Long, Haixia, primary, Jia, Qingzhu, additional, Wang, Liuyang, additional, Fang, Wenfeng, additional, Wang, Zhongyu, additional, Jiang, Tao, additional, Zhou, Fei, additional, Jin, Zheng, additional, Huang, Jiani, additional, Zhou, Li, additional, Hu, Chunyan, additional, Wang, Xinxin, additional, Zhang, Jin, additional, Ba, Yujie, additional, Gong, Yujie, additional, Zeng, Xianghua, additional, Zeng, Dong, additional, Su, Xingxing, additional, Alexander, Peter B., additional, Wang, Li, additional, Wang, Limei, additional, Wan, Yisong Y., additional, Wang, Xiao-Fan, additional, Zhang, Li, additional, Li, Qi-Jing, additional, and Zhu, Bo, additional

Published
2022
Full Text
View/download PDF

Catalog

Books, media, physical & digital resources
See catalog results