2017年6月8日 星期四

aws ebs 好用

可以直接連mount上ec2

2017年5月16日 星期二

website-scraper 好用的爬網站套件

npm install website-scraper

2017年5月7日 星期日

next.j next build失敗 解法

node_module 全部移除 重新安裝
跑一次  node server.js 或 next
再跑 next build
就可以了

解決aws eb multi-docker 資料會被刪除

[最終解法]
eb 只會動 /var/app/current
所以 我直接 mkdir /var/app/db
然後 Dockerrun.aws.json
直接
"volumes": [{
"name": "mongo",
"host": {
"sourcePath": "/var/app/db"
}
}],
這樣就可以了
ebs就沒有很需要了

2017年4月17日 星期一

nginx cache + websocket on AWS single docker

worker_processes 4;

events { worker_connections 1024; }

http {
proxy_cache_path /tmp/cache levels=1:2 keys_zone=cache:30m max_size=1G;

upstream app_upstream {
server localhost:3001;
}

server {
listen 80;

location ~* \.io {
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_set_header X-NginX-Proxy true;

proxy_pass http://app_upstream;
proxy_redirect off;

proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
}

location /static {
proxy_cache cache;
proxy_cache_valid 30m;
proxy_pass http://app_upstream;
}

location / {
proxy_cache cache;
proxy_cache_valid 30m;
add_header X-Cache-Status $upstream_cache_status;
proxy_pass http://app_upstream;
}
}

}

2017年3月20日 星期一

botframewotk locale 設定很特別

export var bot = new builder.UniversalBot(lineConnector, {
    localizerSettings: {
        botLocalePath: __dirname + "/locale",
        defaultLocale: "zh"
    }
});

1. 要注意。tsc 不會自動將 json 引用過去。
2.BotBuilder.json 是 botframework的基礎語言檔 , ex: defualt_number 等,要定義在這裡
3.index.json才是你自已的自定義的 語言檔


2017年3月10日 星期五

docker single docker file on aws nginx setting

FROM node:7.4.0

RUN mkdir -p /usr/src/app
WORKDIR /usr/src/app

ARG NODE_ENV
ENV NODE_ENV $NODE_ENV
RUN npm install yarn -g
ADD package.json /usr/src/app/package.json
RUN yarn

ENV NGINX_VERSION 1.11.8-1~jessie

RUN apt-key adv --keyserver hkp://pgp.mit.edu:80 --recv-keys 573BFD6B3D8FBC641079A6ABABF5BD827BD9BF62 \
    && echo "deb http://nginx.org/packages/mainline/debian/ jessie nginx" >> /etc/apt/sources.list \
    && apt-get update \
    && apt-get install --no-install-recommends --no-install-suggests -y \
                        ca-certificates \
                        nginx=${NGINX_VERSION} \
                        nginx-module-xslt \
                        nginx-module-geoip \
                        nginx-module-image-filter \
                        nginx-module-perl \
                        nginx-module-njs \
                        gettext-base \
    && rm -rf /var/lib/apt/lists/*

# forward request and error logs to docker log collector
RUN ln -sf /dev/stdout /var/log/nginx/access.log \
    && ln -sf /dev/stderr /var/log/nginx/error.log


# Remove the default Nginx configuration file
RUN rm -v /etc/nginx/nginx.conf

# Copy a configuration file from the current directory
ADD nginx.conf /etc/nginx/

# Append "daemon off;" to the configuration file
RUN echo "daemon off;" >> /etc/nginx/nginx.conf

COPY . /usr/src/app/
CMD service nginx start & npm start;
EXPOSE 80

2017年2月14日 星期二

解 dokcer nodejs "ENOENT: no such file or directory, uv_cwd" issue.

可能是docker 內部的神奇部份,
所以只要 node app.js 跟 node ./app.js or node ../app/app.js
每個docker node 錯開 啟使,就正常了

2017年2月11日 星期六

aws multi docker 心得

每次重啟,每一個docker container 的資料會清空,記得要加上volume到外部的folder,才能保存資料。

Dockerrun.aws.json 的 environment 的設定方式

"environment": [
        {
          "name":"PORT",
          "value":"1337"
        },
        {
          "name":"DATABASE_URI",
          "value":"mongodb://mongo:27017/test"
        },
        {
          "name":"APP_ID",
          "value":"test"
        },
        {
          "name":"MASTER_KEY",
          "value":"test_MASTER_KEY"
        },
        {
          "name":"JAVASCRIPT_KEY",
          "value":"test_JAVASCRIPT_KEY"
        }

     
      ]

2017年2月10日 星期五

aws eb multi docker Dockerrun.aws.json

http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/create_deploy_docker_v2config.html

{
  "AWSEBDockerrunVersion": 2,
  "volumes": [
    {
      "name": "app",
      "host": {
        "sourcePath": "/var/app/current/app"
      }
    },

    {
      "name": "nginx-proxy-conf",
      "host": {
        "sourcePath": "/var/app/current/proxy/conf.d"
      }
    },
    {
      "name": "mongo"
    },
    {
      "name": "parse-server"
    }
  ],
  "containerDefinitions": [
    {
      "name": "app",
      "image": "node:latest",
      "essential": true,
      "memory": 256, //不能太少,會 npm install 失敗
      "command": [
        "/bin/bash",
        "/usr/src/app/run-dev.sh"
      ],
      "portMappings": [
        {
          "hostPort": 3000,
          "containerPort": 3000
        }
      ],
      "mountPoints": [
        {
          "sourceVolume": "app",
          "containerPath": "/usr/src/app"
        }
      ]
    },

    {
      "name":"mongo",
      "image":"mongo",
      "essential": true,
      "memory": 128,
      "portMappings": [
        {
          "hostPort": 27017,
          "containerPort": 27017
        }
      ]

    },
    {
      "name": "parse-server",
      "image": "yongjhih/parse-server",
      "essential": true,
      "memory": 256,
      "portMappings": [
        {
          "hostPort": 1337,
          "containerPort": 1337
        }
      ],
      "mountPoints": [
        {
          "sourceVolume": "app",
          "containerPath": "/usr/src/app"
        }
      ],
      "links": [
        "mongo"
      ],
      "environment": [
        {
          "name":"parse-server",
          "PORT": "1337",
          "DATABASE_URI":"mongodb://mongo:27017/test",
          "APP_ID":"test",
          "MASTER_KEY":"test_MASTER_KEY",
          "JAVASCRIPT_KEY":"test_JAVASCRIPT_KEY"

        }
      ]
    },
 
    {
      "name": "nginx-proxy",
      "image": "nginx",
      "essential": true,
      "memory": 128,
      "portMappings": [
        {
          "hostPort": 80,
          "containerPort": 80
        }
      ],
      "links": [
        "parse-server",
        "mongo",
        "app"
      ],
      "mountPoints": [
        {
          "sourceVolume": "awseb-logs-nginx-proxy",
          "containerPath": "/var/log/nginx"
        },
        {
          "sourceVolume": "nginx-proxy-conf",
          "containerPath": "/etc/nginx/conf.d",
          "readOnly": true
        }
      ]
    }

  ]
}

2017年2月8日 星期三

multi docker nginx + nodejs 心得


  1. 用nginx prxoy 可以mapping到不同的image,但外部可以持續用相同的cookie . 
  2. 多專案debug時,直接用內網ip做 proxy_pass就可以了
  3. 用docker-compose做 multi docker,做link後,所有的port都不需要開到外部去,增加安全性.



server {
listen 80;
server_name localhost;
location /test {
proxy_pass http://app:3000/test; # proxy_pass http://192.168.1.100:3000/test;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection 'upgrade';
proxy_set_header Host $host;
proxy_cache_bypass $http_upgrade;
}
location / {
proxy_pass http://app2:3001; # proxy_pass http://192.168.1.100:3001;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection 'upgrade';
proxy_set_header Host $host;
proxy_cache_bypass $http_upgrade;
}
location /parse {
proxy_pass http://parse-server:1337/parse; # proxy_pass http://192.168.1.100:1337/parse;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection 'upgrade';
proxy_set_header Host $host;
proxy_cache_bypass $http_upgrade;
}
} docker-compose.yml <
version: '2'
services:
app:
image: "node:latest"
working_dir: /usr/src/app
command: ./run-dev.sh
volumes:
- ./app:/usr/src/app
environment:
- port=3000
depends_on:
- parse-server
app2:
image: "node:latest"
working_dir: /usr/src/app
command: ./run-dev.sh
volumes:
- ./app:/usr/src/app
environment:
- port=3001
depends_on:
- parse-server
mongo:
image: "mongo"
parse-server:
image: yongjhih/parse-server
# ports:
# - "1337:1337"
environment:
PORT: 1337
DATABASE_URI: "mongodb://mongo:27017/test"
APP_ID: "test"
MASTER_KEY: test_MASTER_KEY
JAVASCRIPT_KEY: test_JAVASCRIPT_KEY
links:
- mongo
depends_on:
- mongo
nginx:
image: nginx
ports:
- "5010:80"
volumes:
- ./proxy/conf.d:/etc/nginx/conf.d
links:
- parse-server
- mongo
- app
- app2

2017年2月7日 星期二

docker nginx 連外面的localhost port

 proxy_pass          http://192.168.1.108:3000/;
直接mapping 內網ip

2017年2月6日 星期一

2017年2月2日 星期四

解injectTapEventPlugin.js 錯誤 ,不要用yarn 用npm install

ERROR in ./~/react-tap-event-plugin/src/injectTapEventPlugin.js
Module not found: Error: Cannot resolve module 'react-dom/lib/EventPluginHub' in /Users/chienhunglin/Dropbox/c
hatbot/web-platform/node_modules/react-tap-event-plugin/src
 @ ./~/react-tap-event-plugin/src/injectTapEventPlugin.js 23:2-41

ERROR in ./~/react-tap-event-plugin/src/TapEventPlugin.js
Module not found: Error: Cannot resolve module 'react-dom/lib/EventConstants' in /Users/chienhunglin/Dropbox/c
hatbot/web-platform/node_modules/react-tap-event-plugin/src
 @ ./~/react-tap-event-plugin/src/TapEventPlugin.js 22:21-60

ERROR in ./~/react-tap-event-plugin/src/TapEventPlugin.js
Module not found: Error: Cannot resolve module 'react-dom/lib/EventPluginUtils' in /Users/chienhunglin/Dropbox
/chatbot/web-platform/node_modules/react-tap-event-plugin/src
 @ ./~/react-tap-event-plugin/src/TapEventPlugin.js 23:23-64

ERROR in ./~/react-tap-event-plugin/src/TapEventPlugin.js
Module not found: Error: Cannot resolve module 'react-dom/lib/EventPropagators' in /Users/chienhunglin/Dropbox
/chatbot/web-platform/node_modules/react-tap-event-plugin/src
 @ ./~/react-tap-event-plugin/src/TapEventPlugin.js 24:23-64

ERROR in ./~/react-tap-event-plugin/src/TapEventPlugin.js
Module not found: Error: Cannot resolve module 'react-dom/lib/SyntheticUIEvent' in /Users/chienhunglin/Dropbox
/chatbot/web-platform/node_modules/react-tap-event-plugin/src
 @ ./~/react-tap-event-plugin/src/TapEventPlugin.js 25:23-64

ERROR in ./~/react-tap-event-plugin/src/TapEventPlugin.js
Module not found: Error: Cannot resolve module 'react-dom/lib/ViewportMetrics' in /Users/chienhunglin/Dropbox/
chatbot/web-platform/node_modules/react-tap-event-plugin/src
 @ ./~/react-tap-event-plugin/src/TapEventPlugin.js 27:22-62

https://github.com/zilverline/react-tap-event-plugin/issues/93
用npm install

2017年1月17日 星期二

噴錯解法 figure out why the markup being generated is different on the client or server:

Warning: React attempted to reuse markup in a container but the checksum was invalid. This generally means that you are using server rendering and the markup generated on the server was not what the client was expecting. React injected new markup to compensate which works but you have lost many of the benefits of server rendering. Instead, figure out why the markup being generated is different on the client or server:
 (client) id="1">
{

     (global as any).navigator.userAgent = req.headers['user-agent'] || 'all';
     (global as any).navigator =  (global as any).navigator || {};
 ..
}

2017年1月14日 星期六

multi docker 部署 aws eb Dockerrun.aws.json

{
  "AWSEBDockerrunVersion": 2, //aws eb docker  版本號
  "volumes": [ //要對應 docker-compose.yml 的 contianer 的 volume設定
    {
      "name": "app-api",
      "host": {
        "sourcePath": "/var/app/current/api" //外部mapping到那裡
      }
    },
    {
      "name": "app-ui",
      "host": {
        "sourcePath": "/var/app/current/ui"
      }
    },
    {
      "name": "nginx-proxy-conf",
      "host": {
        "sourcePath": "/var/app/current/proxy/conf.d"
      }
    }
  ],
  "containerDefinitions": [

    {
      "name": "app-api",
      "image": "node:latest", //不支援 "." ,可是 aws eb 的 single docker 是支援的
      "essential": true,
      "memory": 256, //全部加起來不能超過ec2機子的總量,卡這裡卡很久
      "command": [
        "/bin/bash",
        "/usr/src/app/run-prod.sh"
      ],
      "mountPoints": [
        {
          "sourceVolume": "app-api",
          "containerPath": "/usr/src/app"
        }
      ]
    },

    {
      "name": "app-ui",
      "image": "node:latest",
      "essential": true,
      "memory": 256,
      "command": [
        "/bin/bash",
        "/usr/src/app/run-prod.sh"
      ],
      "portMappings": [
        {
          "hostPort": 3000,
          "containerPort": 3000
        }
      ],
      "mountPoints": [
        {
          "sourceVolume": "app-ui",
          "containerPath": "/usr/src/app"
        }
      ]
    },
    {
      "name":"db",
      "image":"mongo",
      "essential": true,
      "memory": 128,
      "portMappings": [
        {
          "hostPort": 27017,
          "containerPort": 27017
        }
      ]

    },
 
    {
      "name": "nginx-proxy",
      "image": "nginx",
      "essential": true,
      "memory": 128,
      "portMappings": [
        {
          "hostPort": 80,
          "containerPort": 80
        }
      ],
      "links": [
        "app-api",
        "app-ui",
        "db"
      ],
      "mountPoints": [
        {
          "sourceVolume": "awseb-logs-nginx-proxy",
          "containerPath": "/var/log/nginx"
        },
        {
          "sourceVolume": "nginx-proxy-conf",
          "containerPath": "/etc/nginx/conf.d",
          "readOnly": true
        }
      ]
    }

  ]
}

2017年1月9日 星期一

aws eb Multi-container Docker 1.11.2 (Generic) 範例

aws eb
git clone https://github.com/troygoode/docker-compose-and-elastic-beanstalk

Dockerfile nodejs + nginx

nginx.conf
worker_processes 4;

events { worker_connections 1024; }

http {


        server {
            listen 80;
            server_name localhost

            client_max_body_size 1000M;

            access_log            /var/log/nginx/http.log;

            gzip on;
            gzip_disable "msie6";

            gzip_vary on;
            gzip_proxied any;
            gzip_comp_level 6;
            gzip_buffers 16 8k;
            gzip_http_version 1.1;
            gzip_min_length 256;
            gzip_types text/html text/plain text/css application/json application/x-javascript text/xml application/xml application/xml+rss text/javascript application/vnd.ms-fontobject application/x-font-ttf font/opentype image/svg+xml image/x-icon;

            location / {
              proxy_set_header        Host $host;
              proxy_set_header        X-Real-IP $remote_addr;
              proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
              proxy_set_header        X-Forwarded-Proto $scheme;

              proxy_pass          http://localhost:3000;
              proxy_read_timeout  300;
            }
        }

}

========
Dockerfile
FROM node:7.4.0

RUN mkdir -p /usr/src/app
WORKDIR /usr/src/app

ARG NODE_ENV
ENV NODE_ENV $NODE_ENV
RUN npm install yarn -g
ADD package.json /usr/src/app/package.json
RUN yarn

ENV NGINX_VERSION 1.11.8-1~jessie

RUN apt-key adv --keyserver hkp://pgp.mit.edu:80 --recv-keys 573BFD6B3D8FBC641079A6ABABF5BD827BD9BF62 \
 && echo "deb http://nginx.org/packages/mainline/debian/ jessie nginx" >> /etc/apt/sources.list \
 && apt-get update \
 && apt-get install --no-install-recommends --no-install-suggests -y \
      ca-certificates \
      nginx=${NGINX_VERSION} \
      nginx-module-xslt \
      nginx-module-geoip \
      nginx-module-image-filter \
      nginx-module-perl \
      nginx-module-njs \
      gettext-base \
 && rm -rf /var/lib/apt/lists/*

# forward request and error logs to docker log collector
RUN ln -sf /dev/stdout /var/log/nginx/access.log \
 && ln -sf /dev/stderr /var/log/nginx/error.log


# Remove the default Nginx configuration file
RUN rm -v /etc/nginx/nginx.conf

# Copy a configuration file from the current directory
ADD nginx.conf /etc/nginx/

# Append "daemon off;" to the configuration file
RUN echo "daemon off;" >> /etc/nginx/nginx.conf

COPY . /usr/src/app/
CMD service nginx start & npm start;
EXPOSE 80