regex - Creating structured hive table with unstructured GPS packets in csv format -

-

I have a CSV file as outlined below.

VTS, 51,0071,9739 9 65515, nm, gp, inf 01, v, 19, 7 9, 19, 291014, 0000.0000, n, 00000.0000, e, 07 AE VTS, 01.0097 9 7 9 9 65515, SP, GP, 18,07,2253, V, 0000.0000, N, 00000.0000, e, 0.0,0.0,291014,0000,00,4000,11,999,169, B205 VTS, 51, 0071.9739965515, NM, GP, INF0l, V, 18,072311,291014,0000.0000, N, 00000.0000, e, C24E VTS, 01, 0097.9739965515, NM, GP, 19.072311 , V, 0,000.0000, N, 00,000.0000, E, 0.0,0.0,291014,0000,00,4000,11,999,171, B358 VTS, 51,0071,9739965515, knots away, GP, INF01, V, 18,072319,291014,0000.0000, N, 00,000.0000, E, 012F VTS, 51,0071,9739965515, Mono miles away, GP, INF01, V, 19,072326,291014,0000.0000, N, 00,000.0000, E, B2E6 VTS, 0170097, 79, 965515, nm, gp, 18,072326, V, 0000.0000, N, 00000.0000, e, 0.0,0.0,291014,0000,00,4000,11,999,173, EAA 00 VTS, 5l,007l 9 739 9 65 515 nm, GP, INF 01, V, 18,072333,291014, 0000.0000, N, 00000.0000, e 98 9 6 VTS, 5l,007l 9 7 9 9 65 515 nm, GP, INF 01, V, 18,072340,291014,0000.0000, N, 00000.0000, e, 9 B 23

it is to be mapped with the field:

Pkt_header, Gprs_pkt_id, Pkt_length, Sim_no, Msg_id, Gprs_pkt, Gsm_sig_strength, Utc_time, Pkt_validation, latitude, Direction_n_s, longitude, direction_e_w, speed, track_angle, u Tc_date, fuel_adc_values, ignition, odometer_values, supply_int, battery_adc, pkt_id, check_sum

Valid packets are shown in the second field ie GPRS_PKT_ID value 01. The case I used is filtering the CSV data only for valid packets, for which I am using regex, but I am not able to get the full data. Any help would be greatly appreciated

The used hive query is shown below.

Create external table sky_track_testing1 (pkt_header STRING, gprs_pkt_id STRING, pkt_length STRING, sim_no STRING, msg_id STRING, gprs_pkt STRING, gsm_sig_strength STRING, utc_time STRING, pkt_validation STRING, Latitude STRING, STRING longitude STRING, direction_e_w STRING, speed STRING, track_angle STRING, utc_date STRING, fuel_adc_values ​​STRING, ignition STRING, odometer_values ​​STRING, supply_int STRING, battery_adc STRING, pkt_id STRING, check_sum STRING direction_n_s) row format SERDE 'org.apache.hadoop.hive.contrib .serde2.RegexSerDe with SERDEPROPERTIES ("input.regex" = "^ (VTS, 01). * $") Stored as textfile location '/ user / root / sky_track';

This is definitely a wrong query Please help me.

I recommend that you use it for: 

  A = Load '/ user / root / sky_track' (pkt_header, gprs_pkt_id, pkt_length, sim_no, msg_id, gprs_pkt, gsm_sig_strength, utc_time, pkt_validation, latitude, direction_n_, longitude, direction_a_va, speed, track_elegal, utc_date, fuel_adc_value, ignition, Odometer_value, supply_int, battery_adc, pkt_id, check_sum); B = Filter gprs_pkt_id == '01'; Store 'b' / user / root / sky_track_valid ';

Comments

Post a Comment

Popular posts from this blog

apache - 504 Gateway Time-out The server didn't respond in time. How to fix it? -

-
Using a form submission on an embedded iframe , the customer downloads a compressed log file Requested. The request was sent to the server, which is the compressed log files, perform some database operations and returns a compressed file. After just 2 minutes, 504 gateway time-out server did not respond timely message on browser net panel How to fix this error? The log files were taking a long time to compress, and timeout was set to 2min The error was fixed by extending the file file: # # timeout: The number of seconds before getting the time out. # # # Timeout 120 timeout 600
Read more

c# - .net WebSocket: CloseOutputAsync vs CloseAsync -

-
We have a working ASP.NET Web API REST service, which is one of the methods of our controller, HTTPTTEX. .) Socket handler code looks something like this ... Public async task socket handler (AspNetWebSocketContext context) {_webSocket = context.WebSocket; ... while (! Cts.IsCancellationRequested) {WebSocketReceiveResult Results = _webSocket.ReceiveAsync (Input Segment, cts.Token) .Result; WebSocketsStateCollusocketState = _webSocket.State; If (result.MessageType == WebSocketMessageType.Close || currentSocketState == WebSocketState.CloseReceived) {// What should I use. CloseAysnc () or. CloseOutputAsync ()? _webSocket.CloseOutputAsync (WebSocketCloseStatus.NormalClosure, "Client Requested", cts.Token). Wait (); } If (currentSocketState == WebSocketState.Open) {...}}} .What is the difference between .CooseAsync () and CloseOutputAysnc ()? I tried both of them and they both seemed to work fine but some difference should be the same they both describe very similar to...
Read more

c++ - How to properly scale qgroupbox title with stylesheet for high resolution display? -

-
I am trying to apply a stylesheet for QGroupBox (QT4.8), which works on the normal screen ( 96 dpi) high resolution screen (Yoga 2 Pro, 3200x1800, 275 dpi, windows 8.1). The following style looks good on the 275 dpi screen, but the top margins on a regular 96 dpi screen are far too big. QGroupBox {border: 1px solid red; Range radius: 7px; Margin-top: 12x; } QGroupBox :: Title {subcontrol-origin: margin; Subcontrol-position: left above; Padding-left: 10px; Padding-right: 10px; } Changing the top-margin has an effect, but I did not get a proper setting which works on both screens. If I shorten the value, the content of the group box overlaps with the title on 275 dpi screen. I was also playing with units "East", "PX", "MX", "PT". Reading the document I would have guessed, "2 X" was the correct solution, which should be scaled with different screen resolutions. Without the stylesheet, the groupbox works well on both screens. ...
Read more