令牌化字符串，Java中每个令牌的字符数

问题描述

我试图弄清楚是否可以计算每个令牌的字符并显示以下信息：

day被标记，我的输出将是：“ Day有3个字符。”并继续对每个令牌进行操作。

我最后一次打印出每个令牌中的字符数的循环永远不会打印：

public static void main(String[] args) {

    Scanner sc = new Scanner(system.in);

    ArrayList<String> tokenizedInput = new ArrayList<>();
    String sentenceRetrieved;

    // getting the sentence from the user
    System.out.println("Please type a sentence containing at least 4 words,with a maximum of 8 words: ");
    sentenceRetrieved = sc.nextLine();
    StringTokenizer strTokenizer = new StringTokenizer(sentenceRetrieved);

    // checking to ensure the string has 4-8 words
    while (strTokenizer.hasMoretokens()) {
        if (strTokenizer.countTokens() > 8) {
            System.out.println("Please re-enter a sentence with at least 4 words,and a maximum of 8");
            break;

        } else {
            while (strTokenizer.hasMoretokens()) {
                tokenizedInput.add(strTokenizer.nextToken());
            }

            System.out.println("Thank you.");
            break;
        }
    }

    // printing out the sentence
    System.out.println("You entered: ");
    System.out.println(sentenceRetrieved);

    // print out each word given
    System.out.println("Each word in your sentence is: " + tokenizedInput);

    // count the characters in each word
    // doesn't seem to run

    int totalLength = 0;
    while (strTokenizer.hasMoretokens()) {
        String token;
        token = sentenceRetrieved;
        token = strTokenizer.nextToken();
        totalLength += token.length();
        System.out.println("Word: " + token + " Length:" + token.length());
    }

}

}

控制台示例：

请输入至少包含4个单词，最多8个单词的句子：

你好，这是一个测试

谢谢。

您输入了：

你好，这是一个测试

句子中的每个单词都是：[你好，这就是一个测试]

解决方法

首先，我添加了必要的导入并围绕此main方法构建了一个类。这应该编译。

import java.util.ArrayList;
import java.util.Scanner;
import java.util.StringTokenizer;

public class SOQ_20200913_1
{

   public static void main(String[] args) {
   
      Scanner sc = new Scanner(System.in);
   
      ArrayList<String> tokenizedInput = new ArrayList<>();
      String sentenceRetrieved;
   
    // getting the sentence from the user
      System.out.println("Please type a sentence containing at least 4 words,with a maximum of 8 words: ");
      sentenceRetrieved = sc.nextLine();
      StringTokenizer strTokenizer = new StringTokenizer(sentenceRetrieved);
   
    // checking to ensure the string has 4-8 words
      while (strTokenizer.hasMoreTokens()) {
         if (strTokenizer.countTokens() > 8) {
            System.out.println("Please re-enter a sentence with at least 4 words,and a maximum of 8");
            break;
         
         } else {
            while (strTokenizer.hasMoreTokens()) {
               tokenizedInput.add(strTokenizer.nextToken());
            }
         
            System.out.println("Thank you.");
            break;
         }
      }
   
    // printing out the sentence
      System.out.println("You entered: ");
      System.out.println(sentenceRetrieved);
   
    // print out each word given
      System.out.println("Each word in your sentence is: " + tokenizedInput);
   
    // count the characters in each word
    // doesn't seem to run
   
      int totalLength = 0;
      while (strTokenizer.hasMoreTokens()) {
         String token;
         token = sentenceRetrieved;
         token = strTokenizer.nextToken();
         totalLength += token.length();
         System.out.println("Word: " + token + " Length:" + token.length());
      }
   
   }

}

接下来，让我们看一下这个工作示例。看起来直到最后一个while循环（计算字符长度的循环）之前的所有东西都工作正常。但是，如果您注意到，最后一个循环之前的while循环将继续循环，直到没有更多令牌可供提取为止。因此，在完成所有令牌的收集并且没有更多令牌可以收集之后，您尝试创建最终的while循环，要求它收集更多令牌。直到它用完了要收集的令牌之前，它才可能到达while循环！

最后，为了解决这个问题，您可以简单地遍历倒数第二个while循环中添加到的列表，并循环遍历最后一个循环！

例如：

  int totalLength = 0;

  for (String each : tokenizedInput) {

     totalLength += each.length();
     System.out.println("Word: " + each + " Length:" + each.length());

  }