Search

Your search keyword '"GOLDSTEIN, DANIEL"' showing total 3,414 results
Start Over Author "GOLDSTEIN, DANIEL"
3,414 results on '"GOLDSTEIN, DANIEL"'

2. Index

Author

Alonso Bejarano, Carolina, López Juárez, Lucia, Mijangos García, Mirian A., and Goldstein, Daniel M.

Published
2019

3. References

Author

Alonso Bejarano, Carolina, López Juárez, Lucia, Mijangos García, Mirian A., and Goldstein, Daniel M.

Published
2019

5. 2. Journeys toward Decolonizing

Author

Alonso Bejarano, Carolina, López Juárez, Lucia, Mijangos García, Mirian A., and Goldstein, Daniel M.

Published
2019

7. Notes

Author

Alonso Bejarano, Carolina, López Juárez, Lucia, Mijangos García, Mirian A., and Goldstein, Daniel M.

Published
2019

8. Conclusion

Author

Alonso Bejarano, Carolina, López Juárez, Lucia, Mijangos García, Mirian A., and Goldstein, Daniel M.

Published
2019

11. Acknowledgments

Author

Alonso Bejarano, Carolina, López Juárez, Lucia, Mijangos García, Mirian A., and Goldstein, Daniel M.

Published
2019

12. Preface

Author

Alonso Bejarano, Carolina, López Juárez, Lucia, Mijangos García, Mirian A., and Goldstein, Daniel M.

Published
2019

13. “Broken Poem'

Author

Alonso Bejarano, Carolina, López Juárez, Lucia, Mijangos García, Mirian A., and Goldstein, Daniel M.

Published
2019

14. Cover

Author

Ghertner, D. Asher, McFann, Hudson, and Goldstein, Daniel M.

Published
2020

15. Foreword

Author

Ghertner, D. Asher, McFann, Hudson, and Goldstein, Daniel M.

Published
2020

20. Acknowledgments

Author

Ghertner, D. Asher, McFann, Hudson, and Goldstein, Daniel M.

Published
2020

21. Index

Author

Ghertner, D. Asher, McFann, Hudson, and Goldstein, Daniel M.

Published
2020

28. Table of Contents

Author

Ghertner, D. Asher, McFann, Hudson, and Goldstein, Daniel M.

Published
2020

32. GoldFinch: High Performance RWKV/Transformer Hybrid with Linear Pre-Fill and Extreme KV-Cache Compression

Author

Goldstein, Daniel, Obeid, Fares, Alcaide, Eric, Song, Guangyu, and Cheah, Eugene

Subjects
Computer Science - Computation and Language, Computer Science - Artificial Intelligence
Abstract

We introduce GoldFinch, a hybrid Linear Attention/Transformer sequence model that uses a new technique to efficiently generate a highly compressed and reusable KV-Cache in linear time and space with respect to sequence length. GoldFinch stacks our new GOLD transformer on top of an enhanced version of the Finch (RWKV-6) architecture. We train up to 1.5B parameter class models of the Finch, Llama, and GoldFinch architectures, and find dramatically improved modeling performance relative to both Finch and Llama. Our cache size savings increase linearly with model layer count, ranging from 756-2550 times smaller than the traditional transformer cache for common sizes, enabling inference of extremely large context lengths even on limited hardware. Although autoregressive generation has O(n) time complexity per token because of attention, pre-fill computation of the entire initial cache state for a submitted context costs only O(1) time per token due to the use of a recurrent neural network (RNN) to generate this cache. We release our trained weights and training code under the Apache 2.0 license for community use.

Published
2024

33. Eagle and Finch: RWKV with Matrix-Valued States and Dynamic Recurrence

Author

Peng, Bo, Goldstein, Daniel, Anthony, Quentin, Albalak, Alon, Alcaide, Eric, Biderman, Stella, Cheah, Eugene, Du, Xingjian, Ferdinan, Teddy, Hou, Haowen, Kazienko, Przemysław, GV, Kranthi Kiran, Kocoń, Jan, Koptyra, Bartłomiej, Krishna, Satyapriya, McClelland Jr., Ronald, Lin, Jiaju, Muennighoff, Niklas, Obeid, Fares, Saito, Atsushi, Song, Guangyu, Tu, Haoqin, Wirawan, Cahya, Woźniak, Stanisław, Zhang, Ruichong, Zhao, Bingchen, Zhao, Qihang, Zhou, Peng, Zhu, Jian, and Zhu, Rui-Jie

Subjects
Computer Science - Computation and Language, Computer Science - Artificial Intelligence
Abstract

We present Eagle (RWKV-5) and Finch (RWKV-6), sequence models improving upon the RWKV (RWKV-4) architecture. Our architectural design advancements include multi-headed matrix-valued states and a dynamic recurrence mechanism that improve expressivity while maintaining the inference efficiency characteristics of RNNs. We introduce a new multilingual corpus with 1.12 trillion tokens and a fast tokenizer based on greedy matching for enhanced multilinguality. We trained four Eagle models, ranging from 0.46 to 7.5 billion parameters, and two Finch models with 1.6 and 3.1 billion parameters and find that they achieve competitive performance across a wide variety of benchmarks. We release all our models on HuggingFace under the Apache 2.0 license. Models at: https://huggingface.co/RWKV Training code at: https://github.com/RWKV/RWKV-LM Inference code at: https://github.com/RWKV/ChatRWKV Time-parallel training code at: https://github.com/RWKV/RWKV-infctx-trainer

Published
2024

34. Index

Author

Goldstein, Daniel M.

Published
2016

36. Notes

Author

Goldstein, Daniel M.

Published
2016

39. Epilogue

Author

Goldstein, Daniel M.

Published
2016

41. 22.

Author

Goldstein, Daniel M.

Published
2016

Catalog

Books, media, physical & digital resources
See catalog results