doc summarization logs

from https://huggingface.co/spaces/pszemraj/document-summarization
no quantization/optimum
no compiling
the book-summary and booksci models have use_cache set to False (default)

code
relevant code changed here was in summarize.py:
import logging

logging.basicConfig(level=logging.INFO, format="%(asctime)s - %(message)s")

import torch
from tqdm.auto import tqdm
from transformers import AutoModelForSeq2SeqLM, AutoTokenizer


def load_model_and_tokenizer(model_name: str) -> tuple:
    """
    load_model_and_tokenizer - load a model and tokenizer from a model name/ID on the hub
    :param str model_name: the model name/ID on the hub
    :return tuple: a tuple containing the model and tokenizer
    """
    device = "cuda" if torch.cuda.is_available() else "cpu"
    if device == "cuda":
        provider = "CUDAExecutionProvider"
    else:
        provider = "CPUExecutionProvider"

    try:
        import onnxruntime
        from optimum.onnxruntime import ORTModelForSeq2SeqLM

        ort_model = ORTModelForSeq2SeqLM.from_pretrained(
            model_name,
            provider=provider,
            export=True,
        ).to(device)
        tokenizer = AutoTokenizer.from_pretrained(model_name)
        logging.info(f"Loaded model {model_name} to {device} with ONNX Runtime")
        return ort_model, tokenizer

    except ImportError as e:
        logging.info(f"falling back to standard model due to:{e}")
        model = AutoModelForSeq2SeqLM.from_pretrained(
            model_name,
        ).to(device)
        model = model.eval()

        tokenizer = AutoTokenizer.from_pretrained(model_name)

        logging.info(f"Loaded model {model_name} to {device}")
        return model, tokenizer

logs
2023-04-29 22:55:43,348 - INFO - input parameters: {'length_penalty': 0.5, 'repetition_penalty': 1.0, 'no_repeat_ngram_size': 3, 'encoder_no_repeat_ngram_size': 4, 'num_beams': 4, 'min_length': 4, 'max_length': 256, 'early_stopping': True, 'do_sample': False}, batch_length=1024, batch_stride=16

  0%|          | 0/3 [00:00<?, ?it/s]/home/user/.local/lib/python3.8/site-packages/transformers/modeling_utils.py:816: FutureWarning: The `device` argument is deprecated and will be removed in v5 of Transformers.
  warnings.warn(
2023-04-29 22:57:13,693 - INFO - 	In this paper, Turing introduces a new type of question--the "imitation game"--that focuses on the question of whether machines can think. It is a game in which a person, a woman, and a machine compete to guess the answer to a series of questions. The answer is either "X" or "Y" depending on the difficulty of the answer. Turing uses the example of a chess game to illustrate how the question-and-answer method can be used to create a more natural and human-like experience.
Score:	-5.7228

 33%|███▎      | 1/3 [01:30<03:00, 90.34s/it]2023-04-29 22:59:30,670 - INFO - 	In this chapter, the UM uses the analogy of a game to explain why he thinks that machines should be able to perform almost any task that a human can perform. He argues that the only problem with machines is that they cannot be used to perform "impossibility" or "beautiful competitions" because of the way they are made and the fact that they do not have the ability to perform these tasks. The UM also discusses the possibility of using a "thinking machine" to perform the same task. He suggests that the machine should be capable of performing such tasks because the human being is supposed to follow a fixed set of rules. He also suggests that there are "imaginable" machines that would perform such tasks.
Score:	-8.6271

 67%|██████▋   | 2/3 [03:47<01:57, 117.77s/it]2023-04-29 23:00:51,078 - INFO - 	In this chapter, the UM uses the example of a desk machine to explain how a machine works. He describes the three parts of the machine: the store, the executive unit, and the control. The store contains information, which the human machine uses to perform his calculations. The executive part is responsible for seeing that the instructions in the machine are followed.
Score:	-3.7659

100%|██████████| 3/3 [05:07<00:00, 100.71s/it]
100%|██████████| 3/3 [05:07<00:00, 102.57s/it]
2023-04-29 23:00:51,296 - INFO - Runtime: 5.4 minutes
2023-04-29 23:05:34,199 - WARNING - 
        <div style="background-color: #FFA500; color: white; padding: 20px;">
        <h3>Warning</h3>
        <p>Input text was truncated to 2048 words. That's about 17.40% of the submission.</p>
        </div>
        

2023-04-29 23:06:07,965 - INFO - input parameters: {'length_penalty': 0.5, 'repetition_penalty': 1.0, 'no_repeat_ngram_size': 3, 'encoder_no_repeat_ngram_size': 4, 'num_beams': 4, 'min_length': 4, 'max_length': 256, 'early_stopping': True, 'do_sample': False}, batch_length=1024, batch_stride=16

  0%|          | 0/3 [00:00<?, ?it/s]2023-04-29 23:06:54,147 - INFO - 	In this chapter, we're introduced to the "imitation game," a new type of problem that we'll call "the new problem." It's a game in which three people--a man, a woman, and an "interrogator"--play a game of guessing which of them is the man, and which of whom is the woman, by guessing the labels "X" and "Y" on a piece of paper. At the end of this game, the "Interrogator" decides which of the three is the "man" and which is "the woman". The Interrogator has to guess which of these three is "A", and guess the length of the man's or woman's hair by guessing "X's" and guessing "Y's." The third player, "B," is supposed to guess "X-s" as well as "Y-s." She's supposed to pretend to be the woman and guess "A's," but she's not supposed to say "I'm the woman." We're now going to ask, "What happens when a machine is asked to play the role of A in the game?" The answer to this
Score:	-14.1187

 33%|███▎      | 1/3 [00:46<01:32, 46.17s/it]2023-04-29 23:07:21,492 - INFO - 	In this chapter, we are introduced to the concept of "machines." We are told that the machines in the game are intended to be able to think, but that they are not meant to imitate the actions of a human being. In fact, they are meant to carry out actions that could only be carried out by a human. This is because the human is supposed to follow fixed rules, and he has no power to deviate from these rules. The store of the machines is the same as the human's, and the executive unit is responsible for the individual operations that are carried out on the machines.
Score:	-9.0793

 67%|██████▋   | 2/3 [01:13<00:35, 35.10s/it]2023-04-29 23:07:43,880 - INFO - 	In this chapter, we're going to take a look at how a machine's store of rules is replaced by a set of instructions. The instructions are stored in a table called "the table of instructions." The control is supposed to make sure that the instructions are obeyed in the correct order. In other words, the machine should be able to read and write the instructions.
Score:	-6.6151

100%|██████████| 3/3 [01:35<00:00, 29.29s/it]
100%|██████████| 3/3 [01:35<00:00, 31.97s/it]
2023-04-29 23:07:44,144 - INFO - Runtime: 2.17 minutes
2023-04-29 23:08:25,336 - WARNING - 
        <div style="background-color: #FFA500; color: white; padding: 20px;">
        <h3>Warning</h3>
        <p>Input text was truncated to 2048 words. That's about 17.40% of the submission.</p>
        </div>
        

2023-04-29 23:08:42,584 - INFO - Loaded model pszemraj/long-t5-tglobal-base-sci-simplify to cpu
2023-04-29 23:08:42,584 - INFO - input parameters: {'length_penalty': 0.5, 'repetition_penalty': 1.0, 'no_repeat_ngram_size': 3, 'encoder_no_repeat_ngram_size': 4, 'num_beams': 4, 'min_length': 4, 'max_length': 256, 'early_stopping': True, 'do_sample': False}, batch_length=1024, batch_stride=16

  0%|          | 0/3 [00:00<?, ?it/s]2023-04-29 23:08:59,089 - INFO - 	The question, "can machines think?" We propose to consider this question in terms of the question, 'Can machines think?' It is played in a game with three people. The object is to determine whether of the other three is the man or the woman. The game is played with a man, a woman, and a machine. The question is played between two people, the man and the woman, who may be either male or female. The machine is allowed to ask questions to the other two, and the answer is to be written or better still typewritten. It is not possible to play the game between two men and two women, but it is possible for the machine to play it between two women.
Score:	-12.136

 33%|███▎      | 1/3 [00:16<00:32, 16.50s/it]2023-04-29 23:09:14,468 - INFO - 	A machine may be constructed to carry out a particular task, but if it does not carry out this task satisfactorily it cannot be described as a machine. It may be argued that it should be possible to construct a computer which works but which is not capable of carrying out the task that it is supposed to do. This is a strong objection, but it is unlikely to have any great effect on the game.
Score:	-10.2287

 67%|██████▋   | 2/3 [00:31<00:15, 15.84s/it]2023-04-29 23:09:26,551 - INFO - 	A digital computer is a computer which is able to do its calculations in its bead. It is the part of the computer which does these calculations, and the part which is responsible for the control of this computer. The computer's memory is referred to as the store. The control, on the other hand, is regarded as the executive unit. Its function is to see that the computer obeys the instructions supplied to it, and in the correct order.
Score:	-9.7357

100%|██████████| 3/3 [00:43<00:00, 14.12s/it]
100%|██████████| 3/3 [00:43<00:00, 14.65s/it]
2023-04-29 23:09:26,735 - INFO - Runtime: 1.02 minutes
2023-04-29 23:10:52,872 - WARNING - 
        <div style="background-color: #FFA500; color: white; padding: 20px;">
        <h3>Warning</h3>
        <p>Input text was truncated to 2048 words. That's about 17.40% of the submission.</p>
        </div>
        

2023-04-29 23:11:08,258 - INFO - Loaded model pszemraj/long-t5-tglobal-base-sci-simplify-elife to cpu
2023-04-29 23:11:08,258 - INFO - input parameters: {'length_penalty': 0.5, 'repetition_penalty': 1.0, 'no_repeat_ngram_size': 3, 'encoder_no_repeat_ngram_size': 4, 'num_beams': 4, 'min_length': 4, 'max_length': 256, 'early_stopping': True, 'do_sample': False}, batch_length=1024, batch_stride=16

  0%|          | 0/3 [00:00<?, ?it/s]2023-04-29 23:11:29,057 - INFO - 	Computers are able to think, but it is not clear whether they can think in the same way as a human. The question, "can machines think?" can be answered by playing a game called the "imitation game" in which three people are played with a man, a woman, and a machine. The game is played in two rooms, one in the middle of the room, and the other in the front of the other. The player in the game is allowed to ask questions about the other two people, but the machine cannot see or touch the other competitors. The machine must answer these questions in a way that prevents it from being seen or touched by the other players, or hearing their voices. If the machine can think, then it will be possible to use the game to introduce almost all of the fields in which computers can think.
Score:	-12.3093

 33%|███▎      | 1/3 [00:20<00:41, 20.79s/it]2023-04-29 23:11:52,576 - INFO - 	The game is based on the idea that a machine is able to carry out a variety of tasks, but it is not clear whether this machine is capable of carrying out the same tasks as a human machine. It may be suggested that if a computer is unable to perform these tasks, it may not be possible to play the game. This is because it is possible that the computer may not carry out any of the tasks that would normally be carried out by a person. It is also possible that it may be possible for a man to pretend that he is the machine, but if this is not possible, it will be difficult for the machine to perform its tasks. The question is whether the machine can carry out such tasks, and whether it can be constructed in such a way that it would be suitable for playing the imitation game.
Score:	-13.7722

 67%|██████▋   | 2/3 [00:44<00:22, 22.40s/it]2023-04-29 23:12:02,411 - INFO - 	Computers are made up of three parts. The store, executive unit, and control are the three parts that make up a computer. In the computer, the store contains information, and the executive unit contains the instructions that are needed to carry out the calculations. The control, on the other hand, is the part of the computer that makes it possible for the computer to do its calculations. This part is called the control, and is responsible for ensuring that the computer does its calculations correctly. It is also the part that controls how the computer performs its calculations, and so it is important to see that the instructions are correctly carried out.
Score:	-9.8269

100%|██████████| 3/3 [00:54<00:00, 16.66s/it]
100%|██████████| 3/3 [00:54<00:00, 18.05s/it]
2023-04-29 23:12:02,576 - INFO - Runtime: 1.16 minutes
2023-04-29 23:13:47,567 - WARNING - 
        <div style="background-color: #FFA500; color: white; padding: 20px;">
        <h3>Warning</h3>
        <p>Input text was truncated to 2048 words. That's about 17.40% of the submission.</p>
        </div>
        

2023-04-29 23:14:00,293 - INFO - Loaded model pszemraj/long-t5-tglobal-base-16384-booksci-summary-v1 to cpu
2023-04-29 23:14:00,293 - INFO - input parameters: {'length_penalty': 0.5, 'repetition_penalty': 1.0, 'no_repeat_ngram_size': 3, 'encoder_no_repeat_ngram_size': 4, 'num_beams': 4, 'min_length': 4, 'max_length': 256, 'early_stopping': True, 'do_sample': False}, batch_length=1024, batch_stride=16

  0%|          | 0/3 [00:00<?, ?it/s]2023-04-29 23:16:30,190 - INFO - 	In this paper, Turing introduces a new type of question--the "Imitation Game"--that focuses on the question of whether machines can think. Turing explains that this question can be answered by a game called the "imitation game," which is played between two players. The game is played with a man, a woman, and a "interrogator" who stays in the same room as the other two players and asks them to guess which of them is the man or the woman. The answer is either "X" or "Y". The player who answers the question is allowed to play the role of the woman, but he cannot see or touch the other players or hear their voices. This allows Turing to draw a sharp line between physical and intellectual capabilities. The question-and-answer method seems to work well for introducing many of the fields of science, such as chess.
Score:	-10.119

 33%|███▎      | 1/3 [02:29<04:59, 149.89s/it]2023-04-29 23:19:56,785 - INFO - 	In this paper, Dr. Rivers explains how he uses the answer method to introduce almost any field of human endeavor that he wishes to include. He does not want to punish a machine if it is not able to perform a particular task, or a man if he loses at a race. However, if the machine is unable to perform any of these tasks, then the man will lose. If he were to try to pretend that the machine was the machine, he would make a poor showing. If the machine were to perform the same task, the best strategy would be for the man to try and give an answer that would naturally come from a human being. This is a strong objection to the idea that machines should perform something that should be described as "thinking" but which is "very different" from what an individual does. Dr. Murray argues that this is not the case, and that the only thing that a person should do when playing the game is to provide an answer to the question that would be naturally given by the man. This means that if there are no engineers who build the machines, then a team of engineers can construct the machines but
Score:	-14.8491

 67%|██████▋   | 2/3 [05:56<03:03, 183.25s/it]2023-04-29 23:21:10,079 - INFO - 	In this chapter, Dr. Rivers explains how a machine works and how it can be used to perform various tasks. The parts of the machine that are used to do these tasks are called stores, executive units, and control. The store contains information, and the executive unit contains the instructions that the machine uses to perform these tasks. This information is then stored in a table of instructions, which is then arranged in the same way as the instructions in the book of rules.
Score:	-5.637

100%|██████████| 3/3 [07:09<00:00, 133.04s/it]
100%|██████████| 3/3 [07:09<00:00, 143.26s/it]
2023-04-29 23:21:10,240 - INFO - Runtime: 7.38 minutes
2023-04-29 23:25:22,371 - WARNING - 
        <div style="background-color: #FFA500; color: white; padding: 20px;">
        <h3>Warning</h3>
        <p>Input text was truncated to 2048 words. That's about 17.40% of the submission.</p>
        </div>
        
2023-04-29 23:25:24,631 - INFO - Loaded model pszemraj/long-t5-tglobal-base-16384-booksci-summary-v1 to cpu
2023-04-29 23:25:24,631 - INFO - input parameters: {'length_penalty': 0.5, 'repetition_penalty': 1.0, 'no_repeat_ngram_size': 3, 'encoder_no_repeat_ngram_size': 4, 'num_beams': 2, 'min_length': 4, 'max_length': 256, 'early_stopping': True, 'do_sample': False}, batch_length=1024, batch_stride=16

  0%|          | 0/3 [00:00<?, ?it/s]2023-04-29 23:26:03,429 - INFO - 	Turing explains how machines can think in a game called the "imitation game". It is played between three players, a person named A and a female named B. The interquester is allowed to ask questions about the length of the hair of one of the players, and the answer is either "X" or "Y". The answer is "A" if the machine is able to think, and "Y" unless the machine cannot think. Turing then asks if this new question is a worthy question to investigate. This question cuts short the infinite replay of the original question, and allows the machine to be more human.
Score:	-11.2467

 33%|███▎      | 1/3 [00:38<01:17, 38.78s/it]2023-04-29 23:28:02,299 - INFO - 	In this paper, Dr. Rivers argues that the answer method is suitable for bringing about almost any field of human endeavor that we want to include. It does not wish to punish the machine for being unable to shine in Beauty competitions or for losing in aeroplane races. However, if the machine were to be able to perform these tasks, it would be very difficult for the human to perform them. The answer method would be to try to find a way to demonstrate that the machine is capable of performing these tasks. This would be an excellent way to show that the machines are capable of doing these tasks and that they are not a mere imitation of the behavior of a human being. However if there were no way to prove this, the best strategy would be for the machine to provide a solution that would naturally come from a person. This is a strong objection, but at the same time, it does not mean that a machine cannot perform the same task. The question that Dr. Rand focuses on in this paper is "The Machines concerned in the Game". This means that the question is not definite until he has specified what he means by "machine" and how
Score:	-15.9135

 67%|██████▋   | 2/3 [02:37<01:25, 85.89s/it]2023-04-29 23:28:35,992 - INFO - 	In this chapter, the UM uses a table of instructions to explain how a machine works. The UM explains how the instructions are written and how they are organized. The instructions are stored in a small size and are then arranged in the correct order.
Score:	-5.7633

100%|██████████| 3/3 [03:11<00:00, 62.06s/it]
100%|██████████| 3/3 [03:11<00:00, 63.78s/it]
2023-04-29 23:28:36,216 - INFO - Runtime: 3.23 minutes
2023-04-29 23:34:43,755 - WARNING - 
        <div style="background-color: #FFA500; color: white; padding: 20px;">
        <h3>Warning</h3>
        <p>Input text was truncated to 2048 words. That's about 17.40% of the submission.</p>
        </div>
        
2023-04-29 23:34:46,038 - INFO - Loaded model pszemraj/long-t5-tglobal-base-16384-booksci-summary-v1 to cpu
2023-04-29 23:34:46,039 - INFO - input parameters: {'length_penalty': 0.5, 'repetition_penalty': 1.0, 'no_repeat_ngram_size': 3, 'encoder_no_repeat_ngram_size': 4, 'num_beams': 4, 'min_length': 4, 'max_length': 256, 'early_stopping': True, 'do_sample': False}, batch_length=1024, batch_stride=16

  0%|          | 0/3 [00:00<?, ?it/s]2023-04-29 23:37:03,384 - INFO - 	In this paper, Turing introduces a new type of question--the "Imitation Game"--that focuses on the question of whether machines can think. Turing explains that this question can be answered by a game called the "imitation game," which is played between two players. The game is played with a man, a woman, and a "interrogator" who stays in the same room as the other two players and asks them to guess which of them is the man or the woman. The answer is either "X" or "Y". The player who answers the question is allowed to play the role of the woman, but he cannot see or touch the other players or hear their voices. This allows Turing to draw a sharp line between physical and intellectual capabilities. The question-and-answer method seems to work well for introducing many of the fields of science, such as chess.
Score:	-10.119

 33%|███▎      | 1/3 [02:17<04:34, 137.34s/it]2023-04-29 23:40:27,155 - INFO - 	In this paper, Dr. Rivers explains how he uses the answer method to introduce almost any field of human endeavor that he wishes to include. He does not want to punish a machine if it is not able to perform a particular task, or a man if he loses at a race. However, if the machine is unable to perform any of these tasks, then the man will lose. If he were to try to pretend that the machine was the machine, he would make a poor showing. If the machine were to perform the same task, the best strategy would be for the man to try and give an answer that would naturally come from a human being. This is a strong objection to the idea that machines should perform something that should be described as "thinking" but which is "very different" from what an individual does. Dr. Murray argues that this is not the case, and that the only thing that a person should do when playing the game is to provide an answer to the question that would be naturally given by the man. This means that if there are no engineers who build the machines, then a team of engineers can construct the machines but
Score:	-14.8491

 67%|██████▋   | 2/3 [05:41<02:56, 176.42s/it]2023-04-29 23:41:22,649 - INFO - 	In this chapter, Dr. Rivers explains how a machine works and how it can be used to perform various tasks. The parts of the machine that are used to do these tasks are called stores, executive units, and control. The store contains information, and the executive unit contains the instructions that the machine uses to perform these tasks. This information is then stored in a table of instructions, which is then arranged in the same way as the instructions in the book of rules.
Score:	-5.637

100%|██████████| 3/3 [06:36<00:00, 121.20s/it]
100%|██████████| 3/3 [06:36<00:00, 132.20s/it]
2023-04-29 23:41:22,823 - INFO - Runtime: 6.65 minutes
2023-04-29 23:42:36,761 - WARNING - 
        <div style="background-color: #FFA500; color: white; padding: 20px;">
        <h3>Warning</h3>
        <p>Input text was truncated to 2048 words. That's about 17.40% of the submission.</p>
        </div>
        
2023-04-29 23:42:41,232 - INFO - Loaded model pszemraj/long-t5-tglobal-base-16384-book-summary to cpu
2023-04-29 23:42:41,233 - INFO - input parameters: {'length_penalty': 0.5, 'repetition_penalty': 1.0, 'no_repeat_ngram_size': 3, 'encoder_no_repeat_ngram_size': 4, 'num_beams': 2, 'min_length': 4, 'max_length': 256, 'early_stopping': True, 'do_sample': False}, batch_length=1024, batch_stride=16

  0%|          | 0/3 [00:00<?, ?it/s]2023-04-29 23:43:16,833 - INFO - 	In this paper, Turing introduces a new form of problem--the "imitation game"--that uses a criterior to determine whether machines can think. It is a game in which a person, a woman, and an interlocator play a series of questions. The interlocators are given labels and the answers are based on the words that they are asked to respond to. The narrator decides that the answer to "can machines think" will be a combination of words that match the definition of the question.
Score:	-12.2027

 33%|███▎      | 1/3 [00:35<01:11, 35.59s/it]2023-04-29 23:43:56,553 - INFO - 	In this paper, the UM uses the analogy of a game to argue that the game is suitable for almost any field of human endeavor. He does not want to punish the machine for inability to perform its tasks, for its inefficiency to shine in Beauty competitions, or for losing in an aero plane race. He argues that if the game were to be played, the best strategy would be to try to give an answer that would naturally come from a human being. In the end, he concludes that the only thing that can be done to make the game successful is to try and provide an answer to the question of the game that would be naturally given by the human beings.
Score:	-10.5062

 67%|██████▋   | 2/3 [01:15<00:38, 38.02s/it]2023-04-29 23:44:23,255 - INFO - 	In this chapter, we're going to explain how the different parts of a machine work together. The store, executive, and control are all important parts of the machine. For example, if you're doing arithmetic on a desk machine, you might have to do some calculations on that machine as well. The executive part is the part that manages all of this. It's the job of the control, in this case, to make sure that the instructions in the machine are obeying correctly.
Score:	-6.721

100%|██████████| 3/3 [01:42<00:00, 32.85s/it]
100%|██████████| 3/3 [01:42<00:00, 34.00s/it]
2023-04-29 23:44:23,423 - INFO - Runtime: 1.78 minutes