Difference between revisions of "TensorFlow"

From UFRC
Jump to navigation Jump to search
 
(12 intermediate revisions by 5 users not shown)
Line 1: Line 1:
[[Category:Software]][[Category:Library]]
+
[[Category:Software]][[Category:Library]][[Category:Math]]
 
{|<!--CONFIGURATION: REQUIRED-->
 
{|<!--CONFIGURATION: REQUIRED-->
 
|{{#vardefine:app|tensorflow}}
 
|{{#vardefine:app|tensorflow}}
|{{#vardefine:url|https://github.com/tensorflow/tensorflow}}
+
|{{#vardefine:url|https://www.tensorflow.org/}}
 
<!--CONFIGURATION: OPTIONAL (|1}} means it's ON)-->
 
<!--CONFIGURATION: OPTIONAL (|1}} means it's ON)-->
 
|{{#vardefine:conf|}}          <!--CONFIGURATION-->
 
|{{#vardefine:conf|}}          <!--CONFIGURATION-->
 
|{{#vardefine:exe|1}}            <!--ADDITIONAL INFO-->
 
|{{#vardefine:exe|1}}            <!--ADDITIONAL INFO-->
|{{#vardefine:job|}}            <!--JOB SCRIPTS-->
+
|{{#vardefine:job|1}}            <!--JOB SCRIPTS-->
 
|{{#vardefine:policy|}}        <!--POLICY-->
 
|{{#vardefine:policy|}}        <!--POLICY-->
 
|{{#vardefine:testing|}}      <!--PROFILING-->
 
|{{#vardefine:testing|}}      <!--PROFILING-->
Line 18: Line 18:
 
{{App_Description|app={{#var:app}}|url={{#var:url}}|name={{#var:app}}}}|}}
 
{{App_Description|app={{#var:app}}|url={{#var:url}}|name={{#var:app}}}}|}}
  
TensorFlow is an open source software library for numerical computation using data flow graphs. Nodes in the graph represent mathematical operations, while the graph edges represent the multidimensional data arrays (tensors) that flow between them. This flexible architecture lets you deploy computation to one or more CPUs or GPUs in a desktop, server, or mobile device without rewriting code. TensorFlow also includes TensorBoard, a data visualization toolkit.
+
TensorFlow is an open-source software library that is commonly used for implementing artificial neural networks and for deep learning. TensorFlow's numerical computation model is based on data flow graphs. Nodes in the graph represent mathematical operations and the graph edges represent the multidimensional data arrays (tensors) that flow between them. This flexible architecture lets you deploy computation to one or more CPUs or GPUs in a desktop, server, or mobile device without rewriting code. TensorFlow also includes [[Tensorboard]], a data visualization toolkit, and [https://keras.io/ Keras], a high-level (and easier-to-use) neural networks API for Python.
  
 
<!--Modules-->
 
<!--Modules-->
==Required Modules==
+
==Environment Modules==
* {{#var:app}}
+
To use TensorFlow on HiPerGator, you first need to load one of the TensorFlow [[Modules_Basic_Usage|environment modules]].
===Parallel (OpenMP)===
+
Run <code>module spider {{#var:app}}</code> to find out which environment modules are available for TensorFlow.
<!--
+
 
* intel
+
Once you determine which module you want to use, load it as follows (using <code>tensorflow/2.7</code> as an example):
* {{#var:app}}
+
<nowiki>
===Parallel (MPI)===
+
module load tensorflow/2.7</nowiki>
* intel
+
 
* openmpi
 
* {{#var:app}}
 
-->
 
==System Variables==
 
* HPC_{{#uppercase:{{#var:app}}}}_DIR - installation directory
 
 
<!--Configuration-->
 
<!--Configuration-->
 
{{#if: {{#var: conf}}|==Configuration==
 
{{#if: {{#var: conf}}|==Configuration==
Line 40: Line 35:
 
<!--Run-->
 
<!--Run-->
 
{{#if: {{#var: exe}}|==Additional Information==
 
{{#if: {{#var: exe}}|==Additional Information==
 +
As of version 2.0, Keras is packaged with TensorFlow as the <code>tensorflow.keras</code> module. This is the module that you should use. Previously, Keras was developed and distributed separately from TensorFlow; see [https://www.pyimagesearch.com/2019/10/21/keras-vs-tf-keras-whats-the-difference-in-tensorflow-2-0/ keras vs. tf.keras] for details.
  
TensorFlow is installed in a container. Use the 'launch_tensorflow' command to access TensorFlow:
+
To use TensorFlow with a GPU or GPUs on HiPerGator, you must request the <code>--gpus</code> or <code>--gpus-per-task</code> resource and specify the <code>gpu</code> partition in your job script or on the command line as described in the [[GPU_Access|GPU Access]] help page. For example, to start an interactive session with access to a single GPU, you might run the following command.
  $ launch_tensorflow python
+
  <nowiki>
  >>> import tensorflow
+
  srun --partition=gpu --gpus=1 --ntasks=1 --mem=4gb --time=08:00:00 --pty bash -i</nowiki>
or
 
  
$ launch_tensorflow python -c 'import os; import inspect; import tensorflow; print(os.path.dirname(inspect.getfile(tensorflow)))'
+
|}}
 +
<!--Job Scripts-->
 +
{{#if: {{#var: job}}|==Job Script Examples==
 +
To help you get started, here is an example SLURM script for running a Python TensorFlow application on a single GPU on HiPerGator. If you are new to writing SLURM scripts and scheduling SLURM jobs, you will want to first read our help information on [[Getting_Started#Scheduling_jobs_using_SLURM|using SLURM and writing SLURM scripts]]. For information about using GPUs on HiPerGator, please see [[GPU Access]].
  
 +
Note that lines beginning with <code>#SBATCH</code> are instructions to the SLURM scheduler.  Lines beginning with <code># </code> are comments to help you understand the script; feel free to delete them if you adapt this script for your own use.
 +
<div class="mw-collapsible mw-collapsed" style="width:70%; padding: 5px; border: 1px solid gray;">
 +
''Expand this section to view example.''
 +
<div class="mw-collapsible-content" style="padding: 5px;">
 +
<nowiki>
 +
#!/bin/sh
 +
# The job name: you can choose whatever you want for this.
 +
#SBATCH --job-name=my_tensorflow_job
  
 +
# Your email address and the events for which you want to receive email
 +
# notification (NONE, BEGIN, END, FAIL, ALL).
 +
#SBATCH --mail-user=username@ufl.edu
 +
#SBATCH --mail-type=ALL
  
A number of useful python modules have been installed inside the tensorflow container e.g. keras.
+
# The compute configuration for the job. For a job that uses GPUs, the
 +
# partition must be set to "gpu". This example script requests access
 +
# to a single GPU, 16 CPUs, and 30 GB of RAM for a single PyTorch task.
 +
#SBATCH --nodes=1
 +
#SBATCH --partition=gpu
 +
#SBATCH --ntasks=1
 +
#SBATCH --gpus-per-task=1
 +
#SBATCH --cpus-per-task=16
 +
#SBATCH --mem=30gb
  
$ launch_tensorflow python -c 'import keras'
+
# Specifies how long the job will be allowed to run in HH:MM:SS.
|}}
+
#SBATCH --time=05:05:05
<!--Job Scripts-->
+
 
{{#if: {{#var: job}}|==Job Script Examples==
+
# The log file for all job output. Note the special string "%j", which
See the [[{{PAGENAME}}_Job_Scripts]] page for {{#var: app}} Job script examples.
+
# represents the job number.
 +
#SBATCH --output=job_output_%j.out
 +
 
 +
# Prints the working directory, name of the assigned node, and
 +
# date/time at the top of the output log.
 +
pwd; hostname; date
 +
 
 +
module load tensorflow/2.7
 +
 
 +
# This should be the command you would use if you were running your TensorFlow application from the terminal.
 +
python my_tensorflow_script.py
 +
 
 +
date</nowiki>
 +
</div>
 +
</div>
 
|}}
 
|}}
 
<!--Policy-->
 
<!--Policy-->
Line 75: Line 107:
 
<!--Citation-->
 
<!--Citation-->
 
{{#if: {{#var: citation}}|==Citation==
 
{{#if: {{#var: citation}}|==Citation==
If you publish research that uses {{#var:app}} you have to cite it as follows:
+
If you publish research that uses {{#var:app}} you must cite it as follows:
  
 
https://www.tensorflow.org/versions/r0.11/resources/bib.html
 
https://www.tensorflow.org/versions/r0.11/resources/bib.html

Latest revision as of 15:25, 14 December 2022

Description

tensorflow website  

TensorFlow is an open-source software library that is commonly used for implementing artificial neural networks and for deep learning. TensorFlow's numerical computation model is based on data flow graphs. Nodes in the graph represent mathematical operations and the graph edges represent the multidimensional data arrays (tensors) that flow between them. This flexible architecture lets you deploy computation to one or more CPUs or GPUs in a desktop, server, or mobile device without rewriting code. TensorFlow also includes Tensorboard, a data visualization toolkit, and Keras, a high-level (and easier-to-use) neural networks API for Python.

Environment Modules

To use TensorFlow on HiPerGator, you first need to load one of the TensorFlow environment modules. Run module spider tensorflow to find out which environment modules are available for TensorFlow.

Once you determine which module you want to use, load it as follows (using tensorflow/2.7 as an example):

module load tensorflow/2.7


Additional Information

As of version 2.0, Keras is packaged with TensorFlow as the tensorflow.keras module. This is the module that you should use. Previously, Keras was developed and distributed separately from TensorFlow; see keras vs. tf.keras for details.

To use TensorFlow with a GPU or GPUs on HiPerGator, you must request the --gpus or --gpus-per-task resource and specify the gpu partition in your job script or on the command line as described in the GPU Access help page. For example, to start an interactive session with access to a single GPU, you might run the following command.

 srun --partition=gpu --gpus=1 --ntasks=1 --mem=4gb --time=08:00:00 --pty bash -i

Job Script Examples

To help you get started, here is an example SLURM script for running a Python TensorFlow application on a single GPU on HiPerGator. If you are new to writing SLURM scripts and scheduling SLURM jobs, you will want to first read our help information on using SLURM and writing SLURM scripts. For information about using GPUs on HiPerGator, please see GPU Access.

Note that lines beginning with #SBATCH are instructions to the SLURM scheduler. Lines beginning with # are comments to help you understand the script; feel free to delete them if you adapt this script for your own use.

Expand this section to view example.

#!/bin/sh
# The job name: you can choose whatever you want for this.
#SBATCH --job-name=my_tensorflow_job

# Your email address and the events for which you want to receive email
# notification (NONE, BEGIN, END, FAIL, ALL).
#SBATCH --mail-user=username@ufl.edu
#SBATCH --mail-type=ALL

# The compute configuration for the job. For a job that uses GPUs, the
# partition must be set to "gpu". This example script requests access
# to a single GPU, 16 CPUs, and 30 GB of RAM for a single PyTorch task.
#SBATCH --nodes=1
#SBATCH --partition=gpu
#SBATCH --ntasks=1
#SBATCH --gpus-per-task=1
#SBATCH --cpus-per-task=16
#SBATCH --mem=30gb

# Specifies how long the job will be allowed to run in HH:MM:SS.
#SBATCH --time=05:05:05

# The log file for all job output. Note the special string "%j", which
# represents the job number.
#SBATCH --output=job_output_%j.out

# Prints the working directory, name of the assigned node, and
# date/time at the top of the output log.
pwd; hostname; date

module load tensorflow/2.7

# This should be the command you would use if you were running your TensorFlow application from the terminal.
python my_tensorflow_script.py

date


Citation

If you publish research that uses tensorflow you must cite it as follows:

https://www.tensorflow.org/versions/r0.11/resources/bib.html