* Use Runtime Menu to to get high memory

from psutil import virtual_memory
ram_gb = virtual_memory().total / 1e9
print('Your runtime has {:.1f} gigabytes of available RAM\n'.format(ram_gb))

if ram_gb < 20:
  print('Not using a high-RAM runtime')
else:
  print('You are using a high-RAM runtime!')



* *************************************************************

1. Open a new notebook

2. Enable GPU (Edit-->Notebook_settings : GPU Premium High_RAM)

3. !pip install qkeras

4. from google.colab import drive

5. drive.mount('/content/drive')

6. cd /content/drive/MyDrive/pycode

7. exec(open("ppkcode.py").read())




* *************************************************************

1. Open a new notebook

2. Enable GPU

3. Load Drive

from google.colab import drive
drive.mount('/content/drive')

4. Show directory (in new cell)

pwd

5. Change to working directory (in a new cell)

cd /content/drive/MyDrive/pycode

6. Show directory (in a new cell)

7. Run code in a new cell

exec(open("temp.py").read())


* *************************************************************
You can enable GPU with
Edit-->Notebook_settings-->Hardware_accelerator=GPU



import tensorflow as tf
device_name = tf.test.gpu_device_name()
if device_name != '/device:GPU:0':
  raise SystemError('GPU device not found')
print('Found GPU at: {}'.format(device_name))



import tensorflow as tf
import timeit

device_name = tf.test.gpu_device_name()
if device_name != '/device:GPU:0':
  print(
      '\n\nThis error most likely means that this notebook is not '
      'configured to use a GPU.  Change this in Notebook Settings via the '
      'command palette (cmd/ctrl-shift-P) or the Edit menu.\n\n')
  raise SystemError('GPU device not found')

def cpu():
  with tf.device('/cpu:0'):
    random_image_cpu = tf.random.normal((100, 100, 100, 3))
    net_cpu = tf.keras.layers.Conv2D(32, 7)(random_image_cpu)
    return tf.math.reduce_sum(net_cpu)

def gpu():
  with tf.device('/device:GPU:0'):
    random_image_gpu = tf.random.normal((100, 100, 100, 3))
    net_gpu = tf.keras.layers.Conv2D(32, 7)(random_image_gpu)
    return tf.math.reduce_sum(net_gpu)
  
# We run each op once to warm up; see: https://stackoverflow.com/a/45067900
cpu()
gpu()

# Run the op several times.
print('Time (s) to convolve 32x7x7x3 filter over random 100x100x100x3 images '
      '(batch x height x width x channel). Sum of ten runs.')
print('CPU (s):')
cpu_time = timeit.timeit('cpu()', number=10, setup="from __main__ import cpu")
print(cpu_time)
print('GPU (s):')
gpu_time = timeit.timeit('gpu()', number=10, setup="from __main__ import gpu")
print(gpu_time)
print('GPU speedup over CPU: {}x'.format(int(cpu_time/gpu_time)))





* *******************************************


1. In first cell

from google.colab import drive
drive.mount('/content/drive')

import numpy as np
from tensorflow.keras.layers import TextVectorization

2. In second cell (don't seem to need the !, but only one bash command per cell)

   pwd

3. In third cell

   cd /content/drive/MyDrive/pycode

3. In forth cell

   exec(open("example00.py").read())
   



* *******************************************

https://colab.research.google.com/notebooks/gpu.ipynb

* *******************************************

1. Open a new colab notebook

2. load all of your setup into cell and execute it with <ctrl><Enter>

from google.colab import drive
drive.mount('/content/drive')

import numpy as np
from tensorflow.keras.layers import TextVectorization


3. open another new "code" cell


# Example training data, of dtype `string`.
training_data = np.array([["This is the 1st sample."], ["And here's the 2nd sample."]])

# Create a TextVectorization layer instance. It can be configured to either
# return integer token indices, or a dense token representation (e.g. multi-hot
# or TF-IDF). The text standardization and text splitting algorithms are fully
# configurable.
vectorizer = TextVectorization(output_mode="int")

# Calling `adapt` on an array or dataset makes the layer generate a vocabulary
# index for the data, which can then be reused when seeing new data.
vectorizer.adapt(training_data)

# After calling adapt, the layer is able to encode any n-gram it has seen before
# in the `adapt()` data. Unknown n-grams are encoded via an "out-of-vocabulary"
# token.
integer_data = vectorizer(training_data)
print(integer_data)





* *********************************************************************************
* run code in google colabs
* *********************************************************************************

<ctrl> <Enter>  --> runs cell

* * *

from google.colab import drive
drive.mount('/content/drive')

fp = open('/content/drive/MyDrive/pycode/fastfile.txt', 'w')
fp.write("Hello World\n")
fp.write("Thanks world\n")
fp.close()

* * *

!ls drive/MyDrive/pycode

!pwd


* * *

You may want to mount the drive once in a colabs cell, then then put your code into another colabs cell

* * *

import numpy as np
from matplotlib import pyplot as plt

ys = 200 + np.random.randn(100)
x = [x for x in range(len(ys))]

plt.plot(x, ys, '-')
plt.fill_between(x, ys, 195, where=(ys > 195), facecolor='g', alpha=0.6)

plt.title("Sample Visualization")
plt.show()

* * *



* *********************************************************************************

* *********************************************************************************



* *********************************************************************************

sudo pip3 install keras


* *********************************************************************************


https://keras.io/getting_started/intro_to_keras_for_engineers/


import numpy as np
import tensorflow as tf
from tensorflow import keras

convert your data into either NumPy arrarys or tf.data.Dataset objects

build a model for your datasets

use the fit() method to train your model

- - - 

Important concept - neural networks don't process raw data (like text files or images), you must always
convert data to vectorized & standardized representations

  Text files are read into string tensors, then split into words. The words then need to be indexed
  and turned into integer tensors.

  Images need to be read and decoded into integer tensors, then converted to floating point and normalized
  to small values (usually between 0 and 1).

  CSV data needs to be parsed, with numerical features converted to floating point tensors and categorical
  features indexed and converted to integer tensors. Then each feature needs to be normalized and converted
  to zero-mean and unit-variance.

  Etc.

---

Keras models accept three types of inputs:
  NumPy arrays
  TensorFlow Dataset objects
  Python generators (such as subclasses of the keras.utils.Sequence class

---

Keras has a range of utilities to help turn raw data into a Dataset:

  * tf.keras.preprocessing.image_dataset_from_directory

    this assumes you have a directory of images sorted into subdirectories by class
    it will turn this into a labled dataset of image tensors

  * tf.keras.preprocessing.text_dataset_from directory

    this does the same for text files

---

Example - suppose you have something like

main_directory/
  class_a/
    a_image_1.jpg
    a_image_2.jpg
  class_b/
    b_image_1.jpg
    b_image_2.jpg

Then you do this:

    # Create a dataset.
    dataset = keras.preprocessing.image_dataset_from_directory(
      'path/to/main_directory', batch_size=64, image_size=(200, 200))

    # For demonstration, iterate over the batches yielded by the dataset.
    for data, labels in dataset:
       print(data.shape)  # (64, 200, 200, 3)
       print(data.dtype)  # float32
       print(labels.shape)  # (64,)
       print(labels.dtype)  # int32


    (note - you could explicitly specify
      class_names=['class_a', 'class_b'],
    in which cases label 0 will be class_a and 1 will be class_b.)i


Similarly for text files:

    dataset = keras.preprocessing.text_dataset_from_directory(
      'path/to/main_directory', batch_size=64)

    # For demonstration, iterate over the batches yielded by the dataset.
    for data, labels in dataset:
       print(data.shape)  # (64,)
       print(data.dtype)  # string
       print(labels.shape)  # (64,)
       print(labels.dtype)  # int32

---

You can embed the data processing as part of your model.
This is done via preprocessing layers.
  - vectorizing raw strings of text via the TextVectorization layer
  - feature normalization           via the Normalization     layer
  - image rescaling, cropping, or image data augmentation
Some preprocessing layers have state:
  - TextVectorization holds an index mapping word or tokens into integer indices
  - Normalization holds the mean and variance of your features
  (the state of a preprocessing layer is obtained by calling layer.adapt(data))
Example: turning strings into sequences of integer word indices
  from tensorflow.keras.layers import TextVectorization

  # Example training data, of dtype `string`.
  training_data = np.array([["This is the 1st sample."], ["And here's the 2nd sample."]])

  # Create a TextVectorization layer instance. It can be configured to either
  # return integer token indices, or a dense token representation (e.g. multi-hot
  # or TF-IDF). The text standardization and text splitting algorithms are fully
  # configurable.
  vectorizer = TextVectorization(output_mode="int")

  # Calling `adapt` on an array or dataset makes the layer generate a vocabulary
  # index for the data, which can then be reused when seeing new data.
  vectorizer.adapt(training_data)

  # After calling adapt, the layer is able to encode any n-gram it has seen before
  # in the `adapt()` data. Unknown n-grams are encoded via an "out-of-vocabulary"
  # token.
  integer_data = vectorizer(training_data)
  print(integer_data)



* It seems we get a lot of warnings if we don't have a bunch of nvidia gpu stuff (cuda) installed
  We can eliminate most of the warnings with

    export TF_ENABLE_ONEDNN_OPTS=0
    export TF_CPP_MIN_LOG_LEVEL=2