Petrizzo 2024a - Revision history

Tomamil: Tomamil moved page Review 851309404708 to Petrizzo 2024a

2024-06-19T12:28:47Z

Tomamil moved page Review 851309404708 to Petrizzo 2024a

MPetrizzo: Adjusted references and moved table 3 into results.

2024-06-19T01:22:53Z

Adjusted references and moved table 3 into results.

Show changes

MPetrizzo at 20:33, 23 May 2024

2024-05-23T20:33:54Z

MPetrizzo at 20:30, 23 May 2024

2024-05-23T20:30:18Z

MPetrizzo at 20:29, 23 May 2024

2024-05-23T20:29:42Z

MPetrizzo at 20:27, 23 May 2024

2024-05-23T20:27:57Z

MPetrizzo at 20:26, 23 May 2024

2024-05-23T20:26:50Z

MPetrizzo: Added minor heading changes

2024-05-23T20:24:59Z

Added minor heading changes

MPetrizzo: Capitalized Abstract title

2024-05-23T20:23:04Z

Capitalized Abstract title

MPetrizzo: Fixed conversion from docx to Scipedia, moved methodology above Results, added Abstract

2024-05-23T20:22:24Z

Fixed conversion from docx to Scipedia, moved methodology above Results, added Abstract

Show changes

@@ Line 105: / Line 105: @@
 {| style="width: 100%;border-collapse: collapse;"
 |-
-| style="border: 1pt solid black;text-align: center;vertical-align: bottom;" |Rank
+| style="border-top: 1pt solid black;border-right: 1pt solid black;border-left: 1pt solid black;text-align: center;vertical-align: bottom;" |Rank
 | style="border-top: 1pt solid black;border-right: 1pt solid black;text-align: center;vertical-align: bottom;" |Dropout
 | style="border-top: 1pt solid black;border-right: 1pt solid black;text-align: center;vertical-align: bottom;" |Units

@@ Line 105: / Line 105: @@
 {| style="width: 100%;border-collapse: collapse;"
 |-
-| style="border: 1pt solid black;text-align: center;vertical-align: top;" |Rank
+| style="border: 1pt solid black;text-align: center;vertical-align: bottom;" |Rank
 | style="border-top: 1pt solid black;border-right: 1pt solid black;text-align: center;vertical-align: bottom;" |Dropout
 | style="border-top: 1pt solid black;border-right: 1pt solid black;text-align: center;vertical-align: bottom;" |Units

@@ Line 105: / Line 105: @@
 {| style="width: 100%;border-collapse: collapse;"
 |-
-| style="border-left: 1pt solid black;border-right: 1pt solid black;text-align: center;vertical-align: bottom;" |Rank
+| style="border: 1pt solid black;text-align: center;vertical-align: top;" |Rank
 | style="border-top: 1pt solid black;border-right: 1pt solid black;text-align: center;vertical-align: bottom;" |Dropout
 | style="border-top: 1pt solid black;border-right: 1pt solid black;text-align: center;vertical-align: bottom;" |Units

@@ Line 105: / Line 105: @@
 {| style="width: 100%;border-collapse: collapse;"
 |-
-| style="border-top: 1pt solid black;border-right: 1pt solid black;text-align: center;vertical-align: bottom;" |Rank
+| style="border-left: 1pt solid black;border-right: 1pt solid black;text-align: center;vertical-align: bottom;" |Rank
 | style="border-top: 1pt solid black;border-right: 1pt solid black;text-align: center;vertical-align: bottom;" |Dropout
 | style="border-top: 1pt solid black;border-right: 1pt solid black;text-align: center;vertical-align: bottom;" |Units

@@ Line 105: / Line 105: @@
 {| style="width: 100%;border-collapse: collapse;"
 |-
-| style="border-top: 1pt solid black;border-left: 1pt solid black;border-right: 1pt solid black;vertical-align: bottom;" |Rank
+| style="border-top: 1pt solid black;border-right: 1pt solid black;text-align: center;vertical-align: bottom;" |Rank
-| style="border-top: 1pt solid black;border-right: 1pt solid black;vertical-align: bottom;" |Dropout
+| style="border-top: 1pt solid black;border-right: 1pt solid black;text-align: center;vertical-align: bottom;" |Dropout
 | style="border-top: 1pt solid black;border-right: 1pt solid black;text-align: center;vertical-align: bottom;" |Units
 | style="border-top: 1pt solid black;border-right: 1pt solid black;text-align: center;vertical-align: bottom;" |init_lr
-| style="border-top: 1pt solid black;border-right: 1pt solid black;vertical-align: bottom;" |Decay Steps
+| style="border-top: 1pt solid black;border-right: 1pt solid black;text-align: center;vertical-align: bottom;" |Decay Steps
 | style="border-top: 1pt solid black;border-right: 1pt solid black;text-align: center;vertical-align: bottom;" |Decay Rate
 | style="border-top: 1pt solid black;border-right: 1pt solid black;text-align: center;vertical-align: bottom;" |Weight Decay

← Older revision	Revision as of 12:28, 19 June 2024
(No difference)

@@ Line 29: / Line 29: @@
 == MATERIALS AND METHODS ==
 Programming followed a linear creation originating with simple experimentation of BERT encoding and single node dense layers to grid search of multi-layer dense and dropout layers. Versions visibly progressed in both complexities, testing availability, output, and model architecture. Architecture derived from the need of complexity, meaning if the model functioned with high accuracy (95%+) with minimal hidden layers, there was no need to alter this. The data’s split between Depression and Non-Depression was also confirmed (Figure 3).
 <div class="center" style="width: auto; margin-left: auto; margin-right: auto;">[[File:Review_851309404708-image2-c.jpg|centre|thumb|324x324px|('''Figure 3''') – '''Dataset Depression Non-Depression Split''' (Graphic via Student, TensorFlow). Figure representing the split across Depressed and Non-Depressed data samples Final Procedure]]</div>
-Final Procedure
+=== Final Procedure ===
-−
 Modules such as Tensorflow-Text, Keras-Tuner, TensorFlow, and Sklearn were imported to be utilized in the creation of the model. TensorFlow as the main module for the BERT model, Keras-Tuner for grid search, and Sklearn for test-train-split. Data was imported from kaggle/depression-reddit-cleaned as a TensorFlow dataframe, which had two columns; clean_text which contains the textual information of the message and is_depression which contains a binary value representing if the individual has depression. Each textual portion had 2-7 sentences of information which was gathered through multiple Subreddits without documentation from specific locations. Data was already cleaned and ready to be used by the dataset owner. The Sklearn; Test-Train-Validation-Split, of data with 60% used for training, 20% used for testing, and 20% used for validation. Ensuring the same random_state for comparison. Clean_text is normalized through the BERT encoder inside the model architecture, ensuring that each message has 128 tokens and identical tokenization style. The BERT tokenizer and encoder are from the smallest BERT Model; bert_en_uncased_L-2_H-128_A-2 and bert_uncased_preprocess_3. These preprocessors tokenize inputs to 128 tokens, perform lemmatization, and apply padding. The model was created with a dropout layer, hidden layer, dense layer, two activation functions of RELU and Sigmoid, and an output layer. It utilized an AdamW optimizer, which was chosen by a variety of training models, comparing results, and discovery of general machine learning models utilizing AdamW in textual applications. Keras-Tuner grid search was utilized to test random values for dropout percentage, hidden layer node amount, and HyperTuning of AdamW optimizer: such as decay_steps and decay_rate. The HyperTuning values were chosen on variable dependent specifications, such as the necessity of a Dropout, this also determined total training time across all models (Table 1). Most scaling are logarithmic based on the large difference between the minimum and maximum values. Keras-Tuner gathered best validation accuracy from various model performance, and the highest accuracy model is utilized.
@@ Line 94: / Line 93: @@
 |}
-Utilization Procedure
+=== Utilization Procedure ===
-−
 The appendix contains direct code relating to the entirety of the Final Procedure. Inside the code contains comments on both running and testing the model. The entire document must be downloaded into a python file and was intended to be run inside of Google Collab as a .ipynb file. The topmost section contains a link to the original file in Google Collab, and is sectioned into the "Complete Model Training" which imports the dataset, and performs the entire Final Procedure, this section is not necessary to test the model and instead will create a model with identical hyperparameters to the #1 ranked model. The second section is a loading section which will download the trained model as described in this paper, hence skipping the training. The last section allows testing with individual inputs and the entirety of the testing set.

@@ Line 6: / Line 6: @@
 ===
 -->
-== Abstract ==
+== ABSTRACT ==
 Depression is a global mental health issue generalized by a decrease in mood and satisfaction. Treatments for individuals afflicted with depressive symptoms include prescribed medications that require diagnosis to acquire. The purpose of this investigation was to accurately assist psychiatrists in diagnosis procedures to prevent both false positive and false negative conclusions by utilizing machine learning on social media messages. This was done by training a machine learning algorithm which accurately predicted and detected depressive behaviors and communications. As social media messages encompass individual’s general communications among long periods of time with high consistency and frequency, social media messages were used as a method to both train an accurate and consistent machine learning model. Social media messages rely on self reported diagnoses. Based on F1 accuracy normalization across machine learning HyperTuning, average accuracy indicated 97% [+/-0.5%] among a ~7600 sample dataset. Utilizing generalized sentimental analysis has shown less accurate results (~80%) but needs further research.<span id="_heading=h.yuy4vwc7yx7"></span><span id="_heading=h.pfp9y96vqriq"></span>